Method and System for Assessing Vessel Obstruction Based on Machine Learning

ABSTRACT

Methods and systems are described for assessing a vessel obstruction. The methods and systems obtain a volumetric image dataset of a myocardium and at least one coronary vessel, wherein the myocardium comprises muscular tissue of the heart. A three-dimensional (3D) image corresponding to a coronary vessel of interest is created from the volumetric image dataset. Feature data that represents features of both the myocardium and the coronary vessel of interest is generated. At least some of the feature data is determined by a first machine learning-based model based on the 3D image. A second machine learning-based model is used to determine at least one parameter based on the feature data, wherein the at least one parameter represents functionally significant coronary lesion severity of the coronary vessel of interest.

CROSS-REFERENCE TO RELATED APPLICATION(S)

The present application is a continuation of U.S. application Ser. No.16/379,248, filed on Apr. 9, 2019, (Attorney Docket No. PIE-023), whichclaims priority from U.S. Provisional App. No. 62/656,066, filed on Apr.11, 2018, (Attorney Docket No. PIE-023P), both of which are hereinincorporated by reference in their entireties.

BACKGROUND OF THE INVENTION 1. Field

The present application generally relates to methods and systems toassess coronary parameters related to coronary atherosclerosis lesionseverity in one or more coronary arteries.

2. State of the Art

Coronary artery disease (CAD) is one of the leading causes of deathworldwide. CAD generally refers to conditions that involve narrowed orblocked blood vessels that can lead to reduced or absent blood supply tothe sections distal to the stenosis resulting in reduced oxygen supplyto the myocardium, resulting in, for instance, ischemia and chest pain(angina). Narrowing of a blood vessel is called stenosis and is causedby atherosclerosis which refers to the buildup of fats, cholesterol, andother substances in and on vessel walls (plaque), see FIG. 1.Atherosclerotic plaque can be classified according to its components,such as calcified plaque, soft plaque, and mixed plaque, i.e. plaquecontaining calcified and non-calcified components. Such non-calcifiedcomponents include extracellular matrix, smooth muscle cells,macrophages, foam cells, lipid, and fibrous tissue. Calcified plaque isconsidered stable and its amount in the coronary artery is a strongpredictor of cardiovascular events. Unlike calcified plaque,non-calcified plaque and mixed plaque are consider unstable and moreprone to rupture. A plaque rupture may lead to acute major events suchas a stroke, or a heart attached in case the rupture occurs in thecoronary artery. A heart attack can result in a myocardium infarctionresulting in irreversible damage to the myocardium. As different typesof plaque and varying grades of stenosis lead to different patientmanagement strategies, it is important to detect and characterizecoronary artery plaque and stenosis grade.

Besides the grade of stenosis (anatomical stenosis), another veryimportant aspect in the prevention and treatment of CAD is thefunctional assessment of such narrowed anatomical stenosis or blockedblood vessels.

Presently, X-ray angiography is the imaging modality used duringtreatment of stenotic (narrowed) coronary arteries by means of aminimally invasive procedure also known as percutaneous coronaryintervention (PCI) within the catheterization laboratory. During PCI, a(interventional) cardiologist feeds a deflated balloon or other deviceon a catheter from the inguinal femoral artery or radial artery upthrough blood vessels until they reach the site of blockage in theartery. X-ray imaging is used to guide the catheter threading. PCIusually involves inflating a balloon to open the artery with the aim torestore unimpeded blood flow. Stents or scaffolds may be placed at thesite of the blockage to hold the artery open. For intermediate coronaryanatomical lesions (defined as luminal narrowing of 30-70%), forinstance, it is not always obvious if the stenosis is a risk for thepatient and if it is desired to take action. Overestimation of theseverity of the stenosis can cause a treatment which in hindsight wouldnot have been necessary and therefore exposing the patient to risks thatare not necessary. Underestimation of the severity of the stenosis,however, could induce risks because the patient is left untreated whilethe stenosis is in reality severe and actually impedes flow to themyocardium. Especially for these situations it is desired to have anadditional functional assessment to aid in a good decision making.

Fractional Flow Reserve (FFR) has been used increasingly over the last10-15 years as a method to identify and effectively target the coronarylesion most likely to benefit from PCI. FFR is a technique used tomeasure pressure differences across a coronary artery stenosis todetermine the likelihood that the stenosis impedes oxygen delivery tothe heart muscle. The technique involves percutaneously inserting apressure-transducing wire inside the coronary artery and measuring thepressure behind (distal to) and before (proximal to) the lesion and isperformed in the catheterization laboratory. This is best done in ahyperemic state because in the case of maximum hyperemia, blood flow tothe myocardium is proportional to the myocardium perfusion pressure. FFRtherefore provides a quantitative assessment of the functional severityof the coronary lesion as described in Pijls et al. in, “Measurement ofFractional Flow Reserve to Assess the Functional Severity ofCoronary-Artery Stenoses”, N Engl J Med 1996, 334:1703-1708. Althoughthe European Society of Cardiology (ESC) and the American College ofCardiology/American Heart Association (ACC/AHA) guidelines recommend theuse of FFR in patients with intermediate coronary stenosis (30-70%),visual assessment, whether or not supported by QCA, of X-ray coronaryangiograms alone is still used in over 90% of procedures to selectpatients for percutaneous coronary intervention (Kleiman et al,“Bringing it all together: integration of physiology with anatomy duringcardiac catheterization”, J Am Coll Cardiol. 2011; 58:1219-1221). FFR,however, has some disadvantages. The technique is associated with theadditional cost of a pressure wire which can only be used once.Furthermore, measuring FFR requires invasive catheterization with theassociated cost and procedure time. Also, in order to induce (maximum)hyperemia, additional drug infusion (adenosine or papaverine) isrequired, which is an extra burden for the patient.

Coronary CT angiography (CCTA) is a well-established modality foridentification, as well as for exclusion, of patients with suspectedCAD. It allows for noninvasive detection and characterization ofcoronary artery plaque and grading of coronary artery stenosis. Today,these tasks are typically performed in the clinic by visual assessment,or semi-automatically by first utilizing lumen and arterial wallsegmentation and thereafter, defining the presence of plaque orstenosis. However, the former suffers from substantial interobservervariability, even when performed by experienced experts, while thelatter is dependent on coronary artery lumen and wall segmentation whichis typically time-consuming and cumbersome, especially in images withextensive atherosclerotic plaque or imaging artefacts.

Although CCTA can reliably exclude the presence of significant coronaryartery disease, many high-grade stenosis seen on CCTA are not flowlimiting. This potential for false positive results has raised concernsthat widespread use of CCTA may lead to clinically unnecessary coronaryrevascularization procedures. This lack of specificity of CCTA is one ofthe main limitations of CCTA in determining the hemodynamic significanceof CAD (Meijboom et al, “Comprehensive assessment of coronary arterystenoses: computed tomography coronary angiography versus conventionalcoronary angiography and correlation with fractional reserve in patientswith stable angina”, Journal of the American College of Cardiology 52(8) (2008) 636-643). As a result, CCTA may lead to unnecessaryinterventions on the patient, which may pose added risks to patients andmay result in unnecessary health care costs.

To reduce the number of unnecessary catheterization procedures,obtaining coronary artery lesion parameters (such as plaque type,anatomical lesion severity and functional coronary lesion severity)upfront to a catheterization procedures and with a noninvasively imagingmodality such as CCTA is being intensively investigated. Currentlyseveral (semi-)automatic methods for determination of either anatomicalstenosis severity, or the plaque type, or the functional significance ofcoronary artery stenosis in CCTA have been proposed. These methodsheavily rely on the segmentation of the coronary lumen and coronaryvessel wall in case of plaque type detection as for instance describedby Kiris et al. “Standardized evaluation framework for evaluatingcoronary artery stenosis detection, stenosis quantification and lumensegmentation algorithms in computed tomography angiography”, MedicalImage Analysis, vol. 17, no. 8, pp. 859-876, 2013.

Since there are numerous artifacts on CCTA, such as blooming artefactscaused by large arterial calcifications and presents of stents,segmentation inaccuracies are a known problem resulting in accuracy inthe extracted coronary parameters. In addition, motion, lower SNR, andmis-registration reduces its accuracy even more. Therefore, CCTA datawith good image quality is essential for the accuracy of the extractedcoronary parameters such as coronary plaque type, anatomical andfunctional coronary lesion severity.

In Taylor et al “Computational Fluid Dynamics Applied to CardiacComputed Tomography for Noninvasive Quantification of Fractional FlowReserve”, Journal of the American College of Cardiology, Vol. 61, No.22, 2013, and U.S. Pat. No. 8,315,812, a noninvasive method forquantifying FFR from CCTA is described (FFRCT). This technology usescomputational fluid dynamics (CFD) applied to CCTA after semi-automatedsegmentation of the coronary tree including a part of the ascendingaorta covering the region in which both the left coronary artery as wellas the right coronary artery emanate. Three-dimensional (3D) blood flowand pressure of the coronary arteries are simulated, with blood modeledas an incompressible Newtonian fluid with Navier-Stokes equations andsolved subject to appropriate initial and boundary conditions with afinite element method on parallel supercomputer. The FFRCT is modeledfor conditions of adenosine-induced hyperemia without adenosineinfusion. This process is computationally complex and time-consuming andmay require several hours and heavily relies in the 3D anatomicalcoronary model as a result of the segmentation which suffers amongstothers from the same limitation as described above.

Thus, there is a need for obtaining coronary artery lesion parameters(such as plaque type, anatomical lesion severity and functional coronarylesion severity) without relying on the detailed morphology of thecoronary arterial system.

SUMMARY

In accordance with aspects herein, method and systems are provided forassessing a vessel obstruction that involve

a) obtaining a volumetric image dataset of a myocardium and at least onecoronary vessel, wherein the myocardium comprises muscular tissue of theheart;

b) creating a three-dimensional (3D) image corresponding to a coronaryvessel of interest based on the volumetric image dataset of a);

c) generating feature data that represents features of both themyocardium and the coronary vessel of interest, wherein at least some ofthe feature data is determined by a first machine learning-based modelbased on the 3D image of b); and

d) employing a second machine learning-based model to determine at leastone parameter based on the feature data of c), wherein the at least oneparameter represents functionally significant coronary lesion severityof the coronary vessel of interest.

In embodiments, the at least one parameter that represents functionallysignificant coronary lesion severity can be selected from the groupconsisting of: a fractional flow reserve (FFR) value, coronary flowreserve value, instantaneous wave-free ratio, hyperemic myocardiumperfusion value, index of microcirculatory resistance, pressure dropalong a coronary artery, FFR along a coronary artery, and combinationsthereof.

In embodiments, the creation of the 3D image can be based on axialtrajectory of the coronary vessel of interest that is extracted from thevolumetric image dataset of a).

In embodiments, the feature data of c) can be generated by:

using the axial trajectory of the coronary vessel of interest to sampleimage data that corresponds to both the myocardium and the coronaryvessel of interest; and

using the sampled image data as input to the first machinelearning-based model, wherein the first machine learning-based model isconfigured to automatically generate the feature data of c).

In other embodiments, the feature data of c) can be generated by:

using the axial trajectory of the coronary vessel of interest to sampleimage data that corresponds to the coronary vessel of interest;

using the image data as input to the first machine learning-based model,wherein the first machine learning-based model is configured toautomatically generate first feature data, wherein the first featuredata represents features of the coronary vessel of interest;

generating second feature data from additional myocardium analysis ofthe volumetric image dataset, wherein the second feature data representsfeatures of the myocardium; and

combining the first feature data and the second feature data to generatethe feature data of c).

In embodiments, the second feature data can be determined using at leastone of: a convolutional auto-encoder, Gaussian filters, transmuralperfusion ratio, Haralick features, myocardium thickness, and heartshape.

In embodiments, the second feature data can be determined using anunsupervised machine learning which employs a convolution auto-encoder.

In embodiments, the operations of c) and d) can be repeated using imagedata corresponding to the axial trajectory of the coronary vessel ofinterest to determine values of the at least one parameter representingfunctionally significant coronary lesion severity along the length ofthe coronary vessel of interest.

In embodiments, the 3D image can be a multi-planer reformatted (MPR)image that is constructed from samples of the volumetric image datasetalong an axial trajectory of the coronary vessel of interest.

In embodiments, multiple 3D images can be created from the volumetricdataset, wherein the multiple 3D images correspond to a plurality ofcoronary vessels of interest and are based on extraction of axialtrajectories for the plurality of coronary vessels of interest.

In embodiments, the axial trajectories for the plurality of coronaryvessels of interest corresponds to centerlines for plurality of coronaryvessels of interest.

In embodiments, sampled image data from the multiple 3D images can beused as input to the first machine learning-based model to generate datarepresenting features of the plurality of coronary vessels of interest,wherein such data is included in the feature data of c).

In embodiments, the first machine learning-based model can employ arecurrent convolutional neural network or at least one convolutionalautoencoder.

In embodiments, the second machine learning-based model can employ asupervised machine learning classifier.

In embodiments, the second machine learning-based model can employ aneural network, at least one SoftMax classifier, or a support vectormachine.

In embodiments, the second machine learning-based model can be trainedusing reference values derived from invasive measurements involvingpullback of a guidewire located in a target coronary vessel, measuredcoronary flow reserve, index of microcirculatory resistance, majoradverse cardiac events or revascularization after acquisition of thevolume image data set, results of cardiac stress test, and results ofmyocardial imaging by magnetic resonance imaging (MM), SPECT, PET, CTperfusion, or ultrasound.

In embodiments, the training of the second machine learning-based modelcan involve aligning the reference values to spatial coordinates of a 3Dimage corresponding to the target coronary vessel.

In other embodiments, the methods and systems can involve obtaining avolumetric image dataset for a target organ that includes a vessel ofinterest; extracting an axial trajectory extending along of a vessel ofinterest (VOI) within the volumetric image dataset; creating athree-dimensional (3D) multi-planer reformatted (MPR) image based on thevolumetric image dataset and the axial trajectory of the VOI; andextracting a VOI parameter from the MPR utilizing a machinelearning-based vessel obstruction assessment (VOA) model.

Optionally, the methods and systems can implement a prediction phase toat least one of i) detect plaque type, ii) classify anatomical severityof vessel blockage, and/or iii) classify a hemodynamic severity ofvessel obstructions within an unseen portion of the volumetric imagedata set. Optionally, the machine learning-based VOA model generates asequence of cubes from the MPR image, each of the cubes including agroup of voxel from the MPR image, the sequence of cubes created withinsections of the VOI resulting in a sequence of cubes for correspondingsections. Optionally, the machine learning-based VOA model extractsimage features associated with cubes from the sequence of cubesindependently. Optionally, the machine learning analyzes the imagefeatures in sequential dependence. Optionally, a size of the cube isdefined to contain a whole lumen for the VOI and a portion of tissueoutside of the lumen to facilitate extracting the VOI parameter inconnection with positive remodeling, wherein positive remodeling refersto a direction of atherosclerotic plaque growth. Optionally, the axialtrajectory may correspond to a coronary centerline of the VOI, thecoronary centerline representing a center of a coronary lumen along acoronary section of interest, the axial trajectory corresponds to asingle coronary artery, a coronary bifurcation or a full coronary tree,wherein, when the coronary section of interest includes one or morebifurcation(s), the coronary centerline includes the one or morebifurcations.

The machine learning-based VOA model may be based on a recurrentconvolutional neural network (RCNN) which is employed to analyze avicinity along the axial trajectory of the VOI in the MPR image, theRCNN connects a convolutional neural network (CNN) with a recurrentneural network (RNN) connected in series to analyze the portion of theMPR along the axial trajectory as a sequential input. The machinelearning-based VOA model may apply at least one convolution layerfollowed by a max pooling layer to extract an image feature of interestfrom the MPR image and utilizes classifiers for at least one ofcharacterizing plaque type, classify anatomical significance of astenosis or determining the functional significance of a stenosis. Themachine learning-based VOA model may include a feature extraction forcreating a feature vector based on the MPR image, the feature vectorcomprises a series of factors that are measured or extracted from areference database of images, the series of factors describing orcharacterizing the nature of a corresponding wall region of the vesselof interest, the machine learning-based VOA model further including aclassifier to classify the feature vector extracted from the MPR image.The VOI parameter may include at least one of coronary plaque type,anatomical coronary lesion severity or functionally significant coronarylesion severity, and wherein the machine learning-based VOA modelassesses at least one of i) functionally significant coronary lesionseverity, ii) plaque type or iii) anatomical coronary lesion severity.

In accordance with aspects herein, methods and systems are provided totrain a vessel obstruction assessment (VOA) model involving: obtaining atraining database that includes volumetric imaging datasets for multiplepatients and corresponding coronary artery disease (CAD) relatedreference values, the volumetric image data sets being for a targetorgan that includes a vessel of interest, the CAD related referencevalues corresponding to one or more points along a vessel of interestwithin the corresponding imaging data set; for at least a portion of thevolumetric image data sets and corresponding CAD related referencevalues, extracting an axial trajectory extending along of a vessel ofinterest (VOI) within the corresponding volumetric image dataset;creating a three-dimensional (3D) multi-planer reformatted (MPR) imagebased on the corresponding volumetric image dataset and the axialtrajectory of the VOI, the MPR image extending along the axialtrajectory of the VOI; and training a machine learning-based vesselobstruction assessment (VOA) model based on the MPR images, the trainingfurther comprising extracting, from the MPR images, featurescharacterizing a CAD related parameter along the axial trajectory withinthe VOI.

Optionally, the methods and system can align the CAD related referencevalues to spatial coordinates of the corresponding MPR images.Optionally, the methods and systems can generate a sequence of cubesfrom the corresponding MPR images, each of the cubes including a groupof voxel from the corresponding MPR image, the sequence of cubes createdwithin sections of the VOI resulting in a sequence of cubes forcorresponding sections. Optionally, the training further comprisesapplying a convolutional neural network to a sequence of cubes alone theMPR image to build the machine learning-based VOA model. Optionally, theapplying further comprises generating a set of encodings at points alongthe axial trajectory to form a set of one-dimensional (1D) sequences,each of the 1D sequences representing a specific encoding along the VOI,the training further comprising applying a supervised classifier tolearn a fractional flow reserve (FFR) classifier based on the 1Dsequences.

In accordance with aspects herein, a system is provided for assessing avessel obstruction. The system comprises memory configured to store avolumetric image dataset for a target organ that includes a vessel ofinterest; one or more processors that, when executing programinstructions stored in the memory, are configured to: extract an axialtrajectory extending along of a vessel of interest (VOI) within thevolumetric image dataset; create a three-dimensional (3D) multi-planerreformatted (MPR) image based on the volumetric image dataset and theaxial trajectory of the VOI; and extract a VOI parameter from the MPRutilizing a machine learning-based vessel obstruction assessment (VOA)model.

Optionally, the one or more processors are configured to implement aprediction phase to at least one of i) detect plaque type, ii) classifyanatomical severity of vessel blockage, and/or iii) classify ahemodynamic severity of vessel obstructions within an unseen portion ofthe volumetric image data set. Optionally, the machine learning-basedVOA model generates a sequence of cubes from the MPR image, each of thecubes including a group of voxel from the MPR image, the sequence ofcubes created within sections of the VOI resulting in a sequence ofcubes for corresponding sections. Optionally, the machine learning-basedVOA model extracts image features associated with cubes from thesequence of cubes independently. Optionally, the machine learninganalyzes the image features in sequential dependence. A size of the cubemay be defined to contain a whole lumen for the VOI and a portion oftissue outside of the lumen to facilitate extracting the VOI parameterin connection with positive remodeling, wherein positive remodelingrefers to a direction of atherosclerotic plaque growth. The axialtrajectory may correspond to a coronary centerline of the VOI, thecoronary centerline representing a center of a coronary lumen along acoronary section of interest, the axial trajectory may correspond to asingle coronary artery, a coronary bifurcation or a full coronary tree,wherein, when the coronary section of interest includes one or morebifurcation(s), the coronary centerline includes the one or morebifurcations. The machine learning-based VOA model may be based on arecurrent convolutional neural network (RCNN) which is employed toanalyze a vicinity along the axial trajectory of the VOI in the MPRimage, the RCNN connects a convolutional neural network (CNN) with arecurrent neural network (RNN) connected in series to analyze theportion of the MPR along the axial trajectory as a sequential input. Themachine learning-based VOA model may apply at least one convolutionlayer followed by a max pooling layer to extract an image feature ofinterest from the MPR image and utilizes classifiers for at least one ofdetecting plaque type, characterizing plaque type, detecting stenosis ordetermining an anatomical significance of a stenosis. The machinelearning-based VOA model may include a feature extraction for creating afeature vector based on the MPR image, the feature vector comprises aseries of factors that are measured or extracted from a referencedatabase of images, the series of factors describing or characterizingthe nature of a corresponding wall region of the vessel of interest, themachine learning-based VOA model further including a classifier toclassify the feature vector extracted from the MPR image. The VOIparameter may include at least one of coronary plaque type, anatomicalcoronary lesion severity or functionally significant coronary lesionseverity, and wherein the machine learning-based VOA model assesses atleast one of i) functionally significant coronary lesion severity, ii)plaque type or iii) anatomical coronary lesion severity.

In accordance with embodiments herein, a system is provided to train avessel obstruction assessment (VOA) model. The system comprises: memoryconfigured to store a training database that includes volumetric imagingdatasets for multiple patients and corresponding coronary artery disease(CAD) related reference values, the volumetric image data sets being fora target organ that includes a vessel of interest, the CAD relatedreference values corresponding to one or more points along a vessel ofinterest within the corresponding imaging data set; one or moreprocessors that, when executing program instructions stored in thememory, are configured to: for at least a portion of the volumetricimage data sets and corresponding CAD related reference values, extractan axial trajectory extending along of a vessel of interest (VOI) withinthe corresponding volumetric image dataset; create a three-dimensional(3D) multi-planer reformatted (MPR) image based on the correspondingvolumetric image dataset and the axial trajectory of the VOI, the MPRimage extending along the axial trajectory of the VOI; and training amachine learning-based vessel obstruction assessment (VOA) model basedon the MPR images, the training further comprising extracting, from theMPR images, features characterizing a CAD related parameter along theaxial trajectory within the VOI.

Optionally, the processors are further configured to align the CADrelated reference values to spatial coordinates of the corresponding MPRimages. Optionally, the processors are further configured to generate asequence of cubes from the corresponding MPR images, each of the cubesincluding a group of voxel from the corresponding MPR image, thesequence of cubes created within sections of the VOI resulting in asequence of cubes for corresponding sections. Optionally, the processorsare further configured to apply a convolutional neural network to asequence of cubes alone the MPR image to build the machinelearning-based VOA model. Optionally, the one or more processor arefurther configured to generate a set of encodings at points along theaxial trajectory to form a set of one-dimensional (1D) sequences, eachof the 1D sequences representing a specific encoding along the VOI, theone or more processors to perform the training by applying a supervisedclassifier to learn a fractional flow reserve (FFR) classifier based onthe 1D sequences.

BRIEF DESCRIPTION OF THE DRAWINGS

The characteristics of the present application and the advantagesderived therefrom will be more apparent from the following descriptionof non-limiting embodiments, illustrated in the annexed drawings.

FIG. 1 shows an example of coronary atherosclerosis.

FIG. 2 illustrates a flowchart of a machine learning based method fordetermining coronary parameters related to coronary atherosclerosislesion severity in one or more coronary arteries to an embodiment of thepresent application.

FIG. 3 shows a functional block diagram of an exemplary CT system.

FIGS. 4a-4e illustrate the creation of a volumetric MPR image.

FIGS. 5a (i)-5 a(v) show an example on how the results obtained by anembodiment of present application can be presented.

FIG. 5b shows an alternative example on how the results obtained by anembodiment of present application can be presented within a 3D renderedimage.

FIG. 6 shows an example on how the results obtained by an embodiment ofpresent application can be presented in a way that mimics an x-rayangiographic image familiar as being created during a catheterizationprocedure.

FIG. 7 shows a flowchart of the generation of the machine learning RCNNclassification model as performed by the training phase.

FIGS. 8a-8c show a schematic illustration of a co-registration method ofpullback FFR reference values with the CCTA image dataset.

FIGS. 9a-9d show a schematic illustration for the generation of asequence of cubes within an MPR image.

FIG. 10 shows three augmentation methods used for augmentation.

FIG. 11 shows an example of a RCNN architecture to detect coronaryplaque type and anatomical lesion severity along a coronary artery ofinterest.

FIG. 12 shows an example of a RCNN architecture to detect coronaryplaque type, anatomical lesion severity and functionally significant FFRalong a coronary artery of interest.

FIG. 13 shows an example of ischemia due to a stenosis of an artery.

FIG. 14 shows an example of a RCNN architecture combined with amyocardium feature vector to assess functionally significant FFR withina coronary artery.

FIG. 15 shows an alternative approach used to create a sequence of cubeswhich includes myocardium distal to a location of interest.

FIG. 16 shows a flowchart of an embodiment of the present applicationfor the prediction phase.

FIG. 17 shows an alternative workflow to assess the hemodynamicfunctional severity of a coronary artery.

FIG. 18 shows an alternative flowchart of the generation of an FFRclassification model as performed by the training phase.

FIGS. 19a-19c show a visual illustration of the encoder/decoder outputof a CAE illustrated by using an MPR image.

FIG. 20 shows an example of the architecture of a 3D variationalconvolutional autoencoder combined with a 1D auto encoder as used duringlearning FFR classification model, as used during the prediction phase.

FIG. 21a shows an example of the detailed network architecture of the3D-VCAE.

FIG. 21b shows an example of the detailed network architecture of the1D-CAE.

FIG. 22 shows an example of a feature space wherein the training data isrepresented as points, mapped so that the data of the separatecategories are divided by a clear gap that is as wide as possible.

FIG. 23 shows an alternative flowchart of the prediction phase using thetrained FFR classification model as performed by the training phase.

FIG. 24 shows an illustration of the classifier to classify unseen data.Within this visual representation, the input is the feature vectorcomputed from the unseen image and the output two classes.

FIG. 25 shows an alternative flowchart of the generation of amulti-instance learning (MIL) FFR classification model as performed bythe training phase.

FIG. 26 shows an alternative flowchart of the prediction phase using thetrained MIL FFR classification model as performed by the training phase.

FIG. 27 shows an example of obtaining a computed FFR pullback of thecoronary circumflex by using a CAAS Workstation.

FIG. 28 shows a high level block diagram of an example of a CT system.

DETAILED DESCRIPTION OF EMBODIMENTS

The term “unseen”, as used throughout, refers to items which has notbeen used during the training phase. Item in this context means, avolumetric image, a reference value, features, and/or other things usedduring the training phase to train the machine learning based VOA model.Instead, the unseen features, images, geometries, and other unseen itemsrefer to aspects of a patient or object of interest that is beinganalyzed during the prediction phase of operation.

The term “section”, when used in connection with describing cubes ofvoxels, shall refer to substantially continuous geometric regions alonga vessel of interest.

Throughout the present specification, terms which are common in thefield of machine learning/deep learning are used. For detailedexplanation of these terms a reference is made to Litjens et al, “Asurvey on deep learning in medical image analysis”, Med Image Anal. 2017December; 42:60-88.

Throughout the present specification the term “cubes” and “cuboid” areused. Both terms describe a volumetric shape, in which a “cube” means avolumetric cube with equal size of width, height and depth. A “cuboid”is similar to a “cube” but a cuboid may have different sizes of width,height and/or depth.

The present application relates to methods and systems for machinelearning to assess coronary parameters related to CAD such as plaquetype, anatomical severity, and functional severity of one or more vesselobstructions of a target organ based on contrast enhanced volumetricimage dataset. In a preferred embodiment the target organ represent theheart and the vessels the coronary arteries. A functionally significantstenosis is a hemodynamically significant obstruction of a vessel, andwith respect to coronary arteries it defines the likelihood thatcoronary artery obstruction(s) impedes oxygen delivery to the heartmuscle and causes anginal symptoms. Fractional flow reserve is ahemodynamic index for assessment of functionally significant coronaryartery obstruction(s). In addition to fractional flow reserve, otherhemodynamic indices can be used to assess functionally significantcoronary artery obstruction(s), such as coronary flow reserve,instantaneous wave-free ratio, hyperemic myocardium perfusion, index ofmicrocirculatory resistance and pressure drop along a coronary artery.

Embodiments of the present application utilize machine learning todetermine coronary parameters related to CAD such as plaque type,anatomical severity, and functional severity of one or more vesselobstructions from a CCTA dataset. Machine learning is a subfield ofcomputer science that “gives computers the ability to learn withoutbeing explicitly programmed”. Evolved from the study of patternrecognition and computational learning theory in artificialintelligence, machine-learning explores the study and construction ofalgorithms that can learn from and make predictions on data—suchalgorithms overcome following strictly static program instructions bymaking data driven predictions or decisions, through building a modelfrom sample inputs. Machine-learning is employed in a range of computingtasks where designing and programming explicit algorithms is infeasible.

Given a dataset of images with known class labels, machine-learningsystems can predict the class labels of new images. There are at leasttwo parts to any such system. The first part of the machinelearning-based Vessel Obstruction Assessment (VOA) model is a featureextraction (extractor), being an algorithm for creating a feature vectorgiven an image. A feature vector comprises a series of factors (e.g.multiple numbers) that are measured or extracted from the imagedataset(s), which describe or characterize the nature of thecorresponding wall region of the image. These features are then used bythe second part of the VOA model, a classifier, to classify unseenfeature vectors extracted from the unseen image. Given a (large)database of images and extracted feature vectors whose labels are knownand were used beforehand to train the machine learning-based VOA model,classifying unseen images based on the features extracted the same wayas in images with (known) labels (training images) is possible.

FIG. 2 shows a flow chart illustrating the operations according to anembodiment of the present application. The operations employ an imagingsystem capable of acquiring and processing a CCTA dataset of an organ(or portion thereof) or other object of interest. The operations of FIG.2 (as well as the operations of any other methods, algorithms andprocesses described herein) are implemented by one or more processors,while executing program instructions. The one or more processors may beimplemented on various computing devices, such as a smart phone, tabletdevice, laptop computer, desktop computer, workstation, remote server,medical network, and the like. Alternatively, the one or more processorsmay be distributed between one or more separate computing devices, suchthat a portion of the operations are performed by one computing device,while the remainder of the operations are performed by one or more othercomputing devices.

FIG. 3 is a functional block diagram of an exemplary CT system, whichincludes a CT imaging apparatus 112 that operates under commands fromuser interface module 116 and will provide data to data processingmodule 114.

The CT imaging apparatus 112 captures a CT scan of the organ ofinterest. The CT imaging apparatus 112 typically includes an X-raysource and detector mounted in a rotatable gantry. The gantry providesfor rotating the X-ray source and detector at a continuous speed duringthe scan around the patient who is supported on a table between theX-ray source and detector.

The data processing module 114 may be realized by a personal computer,workstation, or other computer processing system. The data processingmodule 114 processes the CT scan captured by the CT imaging apparatus112 to generate data as described herein.

The user interface module 116 interacts with the user and communicateswith the data processing module 114. The user interface module 116 caninclude different kinds of input and output devices, such as a displayscreen for visual output, a touch screen for touch input, a mousepointer or other pointing device for input, a microphone for speechinput, a speaker for audio output, a keyboard and/or keypad for input,etc. The data processing module 114 and the user interface module 116cooperate to carry out the operations of the processes described herein.

The data processing module 114 includes one or more memory 118 and oneor more processors 120. The memory 118 stores, among other things, thecontrast enhanced volume dataset for the target organ, data segments,features extracted from analysis of the data segments, one or more VOAmodels. The memory 118 may also store one or more contrast enhancedvolume datasets for the training organs, CAD related reference values,one or more VOA models, and the like. The memory 118 also storessoftware code that directs the one or more processors 120 to carry outthe operations of the processes described herein. For example, thememory 118 may include an optical disc or other form of persistentmemory such as a USB drive or a network server. The software code can bedirectly loadable into the memory of a data processing module 114 forcarrying out the operations described herein.

In accordance with aspects herein, the imaging system has previouslyacquired and stored at least one CCTA dataset of an object of interest.Any imaging device capable of providing a CT scan can be used for thispurpose. In accordance with aspects herein, the one or more processors120 of the data processing module 114 implement a method for assessing avessel obstruction, the method comprising: obtaining a volumetric imagedataset for a target organ that includes a vessel of interest;extracting an axial trajectory extending along of a vessel of interest(VOI) within the volumetric image dataset; creating a three-dimensional(3D) multi-planer reformatted (MPR) image based on the volumetric imagedataset and the axial trajectory of the VOI; and extracting a VOIparameter from the MPR utilizing a machine learning-based vesselobstruction assessment (VOA) model. In accordance with aspects herein,the one or more processors 120 of the data processing module 114implement A method to train a vessel obstruction assessment (VOA) model,comprising: obtaining a training database that includes volumetricimaging datasets for multiple patients and corresponding coronary arterydisease (CAD) related reference values, the volumetric image data setsbeing for a target organ that includes a vessel of interest, the CADrelated reference values corresponding to one or more points along avessel of interest within the corresponding imaging data set; for atleast a portion of the volumetric image data sets and corresponding CADrelated reference values, extracting an axial trajectory extending alongof a vessel of interest (VOI) within the corresponding volumetric imagedataset; creating a three-dimensional (3D) multi-planer reformatted(MPR) image based on the corresponding volumetric image dataset and theaxial trajectory of the VOI, the MPR image extending along the axialtrajectory of the VOI; and training a machine learning-based vesselobstruction assessment (VOA) model based on the MPR images, the trainingfurther comprising extracting, from the MPR images, featurescharacterizing a CAD related parameter along the axial trajectory withinthe VOI.

The memory 118 may store memory configured to store a volumetric imagedataset for a target organ that includes a vessel of interest.Optionally, the memory 118 may store a training database that includesvolumetric imaging datasets for multiple patients and correspondingcoronary artery disease (CAD) related reference values, the volumetricimage data sets being for a target organ that includes a vessel ofinterest, the CAD related reference values corresponding to one or morepoints along a vessel of interest within the corresponding imaging dataset.

The processors 120 performs extracting a CAD related parameter from theMPR image utilizing a machine learning-based vessel obstructionassessment (VOA) model. The processor 120 implement a training phase totrain a machine learning-based vessel obstruction assessment (VOA) modelbased on the MPR images, the training further comprising extracting,from the MPR images, features characterizing a CAD related parameteralong the axial trajectory within the VOI.

The operations of FIG. 2 can also be carried out by software code thatis embodied in a computer product (for example, an optical disc or otherform of persistent memory such as a USB drive or a network server). Thesoftware code can be directly loadable into the memory of a dataprocessing system for carrying out the operations of FIG. 2.

In this example it is assumed that the imaging system has acquired andstored at least one CCTA dataset of an object of interest. Any imagingdevice capable of providing a CT scan can be used for this purpose.

The present application is particularly advantageous in coronary arterylesion parameters analysis based on CCTA dataset and it will mainly bedisclosed with reference to this field, particularly for patientclassification.

An embodiment of the present application is now disclosed with referenceto FIG. 2. The therein-depicted steps can, obviously, be performed inany logical sequence and can be omitted in parts.

As described in step 201 of FIG. 2, an image dataset is obtained. Suchan image dataset represents a volumetric image dataset for instance asingle contrast enhanced CCTA dataset. This CCTA dataset can be obtainedfrom a digital storage database, such as an image archiving andcommunication system (PACS) or a VNA (vendor neutral archive), a localdigital storage database, a cloud database, or acquired directly from aCT imaging modality. During the CCTA imaging, a contrast agent wasinduced in the patient. Furthermore, the CCTA imaging can be ECGtriggered.

Within step 202 of FIG. 2, the processors extract an axial trajectoryextending along the vessel of interest. For example, the axialtrajectory may correspond to a centerline extending along the vessel ofinterest. When the vessel of interest represents the coronary artery,the axial trajectory may correspond to the coronary centerline, in whichcase the processors extract the coronary centerline. The coronarycenterline represents the center of the coronary lumen along thecoronary section of interest. This can be a single coronary artery, acoronary bifurcation, or the full coronary tree. In case when thecoronary section of interest includes one or more bifurcation(s), thecoronary centerline will include bifurcation(s) as well but not its sidebranch. As described further by step 203, the centerline is used tocreate the MPR image. In case bifurcation and/or the coronary tree isanalyzed, multiple centerline are extracted, as for example two coronarycenterlines are extracted when analyzing one bifurcation; one coronarycenterline identified by a proximal location to a distal location withinthe main branch of bifurcation, and one centerline identified by aproximal location to a distal location within the side branch ofbifurcation. For the purpose of current application, it is not requiredthat the extracted coronary centerline represents the center of thecoronary lumen accurately. A rough estimation of the coronary centerlineis sufficient, although the coronary centerline should not exceed thecoronary lumen. The extraction of the coronary centerline can beperformed manually or (semi)automatically. An example of a semiautomaticapproach is described by Metz et al., “Semi-automatic coronary arterycenterline extraction in computed tomography angiography data”,proceedings/IEEE International Symposium on Biomedical Imaging: fromnano to macro, May 2007. An example of an automatic coronary centerlineextraction method is described by Wolterink et al. in which machinelearning is utilized to automatically extract the coronary centerline in“Coronary artery centerline extraction in cardiac CT angiography using aCNN-based orientation classifier”, Med Image Anal. 2019 January;51:46-60. The method extracts, after placement of a single seed point inthe artery of interest, the coronary centerline between the ostium andthe most distal point as visualized in the CCTA image dataset.

Within step 203 of FIG. 2, a three-dimensional (3D) multi-planarreformatted (MPR) image is created of the coronary artery of interest.FIGS. 4a-4d provide an illustration of the creation of a volumetric (3D)MPR image, further called an MPR image. Image 401 of FIG. 4a shows avolumetric rendering of a CCTA dataset (FIG. 2, 201), in which the rightcoronary artery 402 is selected as an example to create an MPR image.With respect to MPR, there is a distinction between straight MPR andcurved MPR. Both for the straight MPR as well for the curved MPR theextracted axial trajectory (e.g. centerline) is used to create isotropicMPR image from the obtained image dataset 201. The resolution of the MPRimage is predefined and is for example 0.3 mm. The MPR image can also becreated in a non-isotropic way.

A straight MPR reformats the image towards a cuboid image 403 along theextracted axial trajectory (e.g. coronary centerline) 402 in such a waythat the coronary centerline is in the center of the cuboid 404. Image403 of FIG. 4b illustrates the cuboid resampled image (straight MPR) andone ‘slice’ is visualized within the cuboid resampled image for easyinterpretation. Image 405 of FIG. 4c shows a one ‘slice’ of the sameresampled image, but the visualization plane is rotated around thecenterline 404, to illustrate the visualization of the coronarybifurcation (406) within the extracted right coronary artery. A curvedMPR image is reconstructed along the curved course of the coronarycenterline. Images 408 a and 408 b of FIGS. 4d and 4e shows two examplesof a curved MPR image and visualized as a single ‘slice’, in which theslice orientation refers to a curved plane which can be rotated alongthe curved coronary artery. Again, this is just for visualizationpurpose, the application will use the full 3D straight or curved MPRimage. The advantage of a curved MPR image is that the curvature ortortuosity of the extracted centerline can be taken into account withinthe described machine learning network architectures within thisapplication.

Within step 204 of FIG. 2, the coronary parameters are extracted byutilizing a machine learning based Vessel Obstruction Assessment (VOA)model. Within current application, coronary plaque type, anatomicalcoronary lesion severity and functionally significant coronary lesionseverity are assessed by utilizing machine learning based VOA models onthe MPR image as a result of step 203 of FIG. 2. The machine learningapproach for the assessment of functionally significant coronary lesionseverity is slightly different as compared to the machine learningmethod for the detection of plaque type and anatomical coronary lesionseverity and is further described by the corresponding flow chartswithin current application. Moreover, a distinction is made between thelearning phase and the prediction phase within current application andis described in more detail in the description of the different machinelearning methods. Within the learning phase (or training phase), themachine learning model is trained. This involves training the parametersof the selected model, or also called the network architecture, by usingreference standard (205 of FIG. 2). The reference standard is a databasewhich contains data of multiple patients. Each set within the databasecontains for each patient a) contrast enhanced CT image datasets (201represents reference image sets during the training phase) andcorresponding b) CAD related reference values. For example, the CADrelated reference values may represent at least one of the plaque type,anatomical stenosis severity, invasively measured fractional flowreserve, and/or other hemodynamic indices. As a further example, thehemodynamic indices can represent indices used to assess functionallysignificant coronary artery obstruction(s) such as coronary flowreserve, instantaneous wave-free ratio, hyperemic myocardium perfusion,index of microcirculatory resistance and pressure drop along a coronaryartery.

After training the machine learning model, step 204 of FIG. 2 isconfigured to predict the coronary plaque type, and/or anatomicalstenosis severity and/or the functional significance of the coronary ofinterest based on analysis of the MPR image as a result of step 204.Within the prediction phase, unseen image data is used (201) and step205 of FIG. 2 is detached.

The output (step 206 of FIG. 2) is a prediction of the coronary plaquetype, and/or anatomical stenosis severity and/or the functionalsignificance of lesion(s) within the coronary of interest. This resultcan be presented to the user in various ways. FIGS. 5a (i)-5 a(v) andFIG. 5b shows some examples of the presentation of the results to theuser. Image 501 of FIG. 5a (i) represents the MPR image as a result ofstep 203 of FIG. 2. Image 502 of FIG. 5a (ii) shows the plaque typeclassification as a color or grey value superimposed on the MPR image,in which the colors represent different plaque types (e.g. no-plaque,calcified plaque, non-calcified plaque or mixed plaque). Image 503 ofFIG. 5a (iii) shows the anatomical stenosis severity superimposed as acolor or grey value on the MPR image, in which the colors representdifferent labels of anatomical stenosis severity. Within Image 503,three anatomical stenosis severity classes are visualized, no-stenosis,non-anatomical significant stenosis (with <50% luminal narrowing) oranatomical significant stenosis (with >50% luminal narrowing). Image 504of FIG. 5a (iv) shows the FFR value along the MPR image, in which they-axis represents the estimated FFR value and the x-axis the locationalong the length of the coronary of interest which corresponds to thex-axis of the MPR image 501. Various other ways of visualization can beused, for instance the results can be superimposed on the curved MPR, onthe orthogonal views or the results can be visualized on the volumetricrendering of the image dataset (201 of FIG. 2) as shown by image 505 ofFIG. 5a (v) in which 506 illustrates a color-coded result of forinstance the FFR value along the coronary centerline. Note that in Image505, the coronary centerlines 506 are visualized to appeal the coronarylumen by using volume rendering techniques. FIG. 5b shows anotherexample of visualization the results. Image 507 is a volumetricrendering of only the coronary arteries including a small part theascending aorta in which the left and right coronary artery emanate fromthe aorta. Superimposed on the rendered coronary arteries within 507, acolor or grey value is visualized (509) which represents for instancethe numerical FFR value and 508 provides the color legend to map thecolor or grey value to a (numerical) value. Another visualizationapproach is presented by FIG. 6. In this figure the results are shownwithin a simulated angio view. A simulated angio view is an image whichmimics an x-ray angiographic image of the coronaries as view by aparticular angulation of the c-arm, as for instance disclosed by U.S.Pat. No. 9,008,386B2 and U.S. Ser. No. 10/192,352B2. Image 601 of FIG. 6shows such a simulated angio view in which the angulation (602, rotationangle of the c-arm and angulation rotation angle of the c-arm) can becontrolled by the user. 603 shows the extracted centerline as a resultof step 202 of FIG. 2, on which the color or grey value represent forinstance the FFR value and 604 provides the color legend to map thecolor or grey value to a (numerical) value.

FIG. 7 illustrates a framework for training the machine learning modelto extract coronary parameters within the MPR image as described by step204 of FIG. 2. FIG. 7 illustrates the training phase of the system todetect coronary plaque type, anatomical stenosis severity andfunctionally significant coronary stenosis severity. In step 701 of FIG.7 the reference standard is obtained as used to train the machinelearning model. For example, the processor may obtain a trainingdatabase that includes volumetric imaging datasets for multiple patientsand corresponding coronary artery disease (CAD) related referencevalues. The volumetric image data sets may be for a target organ thatincludes a vessel of interest. The CAD related reference valuescorresponding to one or more points along a vessel of interest withinthe corresponding imaging data set. The reference standard is a databasewhich contains data of multiple patients. Each set within this databasecontains for each patent a) contrast enhanced CT datasets step 703 andcorresponding b) CAD related reference values. For example, the CADrelated reference values may represent at least one of the plaque type,anatomical stenosis severity, invasively measured fractional flowreserve, and/or other hemodynamic indices. As a further example, thehemodynamic indices can represent indices used to assess functionallysignificant coronary artery obstruction(s) such as coronary flowreserve, instantaneous wave-free ratio, hyperemic myocardium perfusion,index of microcirculatory resistance and pressure drop along a coronaryartery.

The reference plaque type contains annotation of different types ofcoronary plaque such as no-plaque, calcified plaque, non-calcifiedplaque, or mixed plaque.

The reference of anatomical stenosis severity contains differentstenosis grades, for example grade 0 (no luminal narrowing), grade 1(0-25% luminal narrowing), grade 2 (25-50% luminal narrowing), grade 3(50-75% luminal narrowing) grade 4 (75-100% luminal narrowing).

Within current application the invasively measured fractional flowreserve is preferably measured along the coronary of interest, resultingin an invasively measured fractional flow reserve value at each positionalong the coronary centerline. This can be obtained by performing apullback during the measurement of the fractional flow reserve. In thecatheterization laboratory, the interventional cardiologist or physicianplaces the FFR wire at the distal location within the coronary ofinterest. During automatic or manual pullback, the FFR value iscontinuously measured until the FFR wire reaches the coronary ostium.

In step 704 of FIG. 7, the processor extracts an axial trajectoryextending along of a vessel of interest (VOI) within the correspondingvolumetric image dataset. For example, the coronary centerline may beextracted which represent the center of the coronary lumen along thecoronary section of interest. This step may be implemented in a mannersubstantially similar to step 202 of FIG. 2.

In step 705 of FIG. 7, the processor creates a three-dimensional (3D)multi-planer reformatted (MPR) image based on the correspondingvolumetric image dataset and the axial trajectory of the VOL The MPRimage extends along the axial trajectory of the VOL For example, amulti-planer reformatted (MPR) image may be created along the extractedcoronary section as a result from step 704. Step 705 of FIG. 7 may beimplemented in a manner substantially similar to step 203 of FIG. 2.

In step 706 of FIG. 7, the operations ensure that the CAD-relatedreference values 702 are aligned to the spatial coordinates of the MPRimage. In case the CAD-related reference values (e.g. manual annotationof plaque type, anatomical lesion severity, functional lesion severitysuch as for instance FFR) are obtained by using the MPR image as aresult from step 705, this step may be skipped. When the CAD-relatedreference values are obtained, for instance by annotation usingorthogonal views of the contrast enhanced CT datasets (step 703), thisstep transforms the annotation into the MPR view. Such a transformationis performed by using the extracted centerline as a result of step 704.

To ensure that the fractional flow reserve values along the coronaryartery as measured in the catheterization laboratory (e.g., pullback FFRreference values) are aligned to the spatial coordinates of the MPRimage, a co-registration is performed between the image dataset 703 andthe invasively measured pullback FFR. To allow co-registration of thepullback FFR measurement with the CT dataset, pullback motioninformation is obtained indicative of the pullback rate or speed duringwithdrawal of the FFR wire from an FFR wire start location (e.g., distalposition in the coronary artery) to an FFR wire end location (e.g.,proximal position in the coronary artery or the ostium of the coronaryartery). The pullback motion information can be obtained by measuringthe longitudinal motion of the FFR wire during pullback. The measurementmay be obtained in various manners, such as by means of a motionmeasurement system, or for instance by utilizing a motorized pullbackdevice that maintains a constant pullback speed. The one or moreprocessors of the system utilize the time required to pullback the FFRwire and the pullback speed to calculate a length of a pullbackdistance. In order to align the pullback FFR reference values into theMPR image, the one or more processors transform the length of thepullback distance to the image dataset used in 703.

FIGS. 8a-8c , collectively, provide a schematic illustration of aco-registration method of a pullback FFR reference value (e.g., pullbackdistance) with the CCTA image dataset. Image 801 of FIG. 8a shows anx-ray coronary angiographic image as acquired within the catheterizationlaboratory. Image 802 of FIG. 8b shows a volume rendered CCTA imagebelonging to the same patient. Image 806 of FIG. 8c shows an x-rayfluoroscopic image without contrast liquid present.

The x-ray coronary angiographic image 801 of FIG. 8a illustrates theright coronary artery with an FFR pressure wire inserted therein to adesired distal position at which a pressure sensor may obtainfirst/distal pressure measurements of interest. The dot 803 indicates alocation of the pressure sensor on the FFR pressure wire when located atthe distal position within the coronary artery before pullback on thex-ray angiographic image. The position denoted by dot 803 may also bereferred to as a distal pressure sensor position. The distal position ofthe pressure sensor (and the entire FFR pressure wire) is easilyidentifiable on x-ray fluoroscopic image (without contrast liquidpresent, 806) due to the radiopaque marker on the FFR wire 807 (as shownin the image 806 of FIG. 8c ) enabling to localize the pressure sensoron the FFR wire.

Image 802 of FIG. 8b shows a volume rendered CCTA image (belonging tothe same patient in which the pullback FFR reference values areobtained). In the image 802, the right coronary artery 804 is identifiedfor instance as a result from step 704 of FIG. 7. Co-registration of thepullback FFR reference values is performed by identifying the locationof the FFR pressure wire before pullback within the CCTA dataset (805),for instance manually identifying supported by anatomical landmarks suchas bifurcation location, and align the FFR values by matching thelengths (length of the 3D extracted centerline with the length of theFFR pullback). The identification of the location of the FFR pressurewire before pullback within the CCTA dataset can also be performed byregistration of the x-ray angiographic image with the CCTA dataset, asfor instance by using the method of Baka et al. “Oriented GaussianMixture Models for Nonrigid 2D/3D Coronary Artery Registration”, IEEETrans Med Imaging. 2014 May; 33(5):1023-34. Baka et al describes amethod to register a 2D x-ray angiographic image to a 3D volumetricimage dataset (CCTA) by using a Gaussian mixture model (GMM) basedpoint-set registration technique. Since the location of the pressuresensor (807) on the FFR wire can be easily performed using x-rayfluoroscopic image (806) by means of image processing techniques, thetransformation of this location into the CCTA image data (805) isstraightforward using the deformation field resulting from the 2D/3Dregistration as described by Baka et al.

Returning to FIG. 7, in step 707, the processors train a machinelearning-based vessel obstruction assessment (VOA) model based on theMPR images. Among other things, the training comprises extracting, fromthe MPR images, features characterizing a CAD related parameter alongthe axial trajectory within the VOL For example, the CAD relatedparameter values may represent at least one of the plaque type,anatomical stenosis severity, invasively measured fractional flowreserve, and/or other hemodynamic indices. As a further example, thehemodynamic indices can represent indices used to assess functionallysignificant coronary artery obstruction(s) such as coronary flowreserve, instantaneous wave-free ratio, hyperemic myocardium perfusion,index of microcirculatory resistance and pressure drop along a coronaryartery.

For example, in connection with the training, the processors generate asequence of cubes from the MPR image which are used to train the networkas described further in step 709. A sequence of cubes is created withinsections were the CAD-related reference value is available as a resultfrom step 706, resulting in a sequence of n cubes for each section. Theterm section in this context refers to a continuous part within the MPRimage in which the CAD-related reference value is available and not aspecific anatomical segment of the coronary artery. This allows that theCAD-related reference value do not need to cover the full length of theMPR image. For instance, the annotation of plaque types or grade ofcoronary luminal narrowing needs only to be present in coronary regionsin which there truly is a coronary plaque. In regions were no annotationof plaque type or lesion obstruction is available, the system can eitherautomatically mark these regions as ‘no coronary plaque’ or ‘noanatomical coronary obstruction present’ or ignore them in the trainingphase.

FIGS. 9a-9d , collectively, provide a schematic illustration of aprocess to generate the sequence of cubes within a section. Image 901 ofFIG. 9a shows a 3D view of the heart. Image 902 of FIG. 9b is azoomed-in view of the rectangle within image 901. Within image 902, theextracted axial trajectory (in the example a coronary centerline tree)is shown (904). The coronary centerline exists of a number of points(907). Image 903 of FIG. 9c shows the straight MPR image (as a result ofstep 704 of FIG. 7) created using the centerline 904. FIG. 9d shows asection (906) of image 903 where a sequence of cubes is created whichcovers a small part of the MPR image. The size of the cubes is definedso that it contains the whole arterial lumen and the vicinity of theartery that may be needed in case of positive remodeling. Positiveremodeling refers to the outwards direction of atherosclerotic plaquegrowth. CAD is currently defined as clinically significant when luminalnarrowing is present, typically at the 50% diameter reduction threshold.However, in early atherosclerosis the first arterial changes include ofcompensatory enlargement of both the outer wall of the vessel as well asthe lumen, termed compensatory enlargement or positive remodeling. Eachcube is for instance 25×25×25 voxels, but other cube sizes are possible.The distance between the cubes (stride) is m centerline points (m is forinstance 5 centerline points in case the distance between the centerlinepoints is maximum 1 voxel, but other stride values are possible). Themaximum number of cubes is defined by the amount of centerline points,or length, of the MPR image. In a preferred embodiment the maximumnumber of cubes is limited to the longest annotated plaque type section,which is for instance 25. In case multiple sections are present in theMPR image, also multiple (n) sequence of cubes are generated.

Returning to FIG. 7, in step 708, the training data is augmented. Priorto training several data augmentation techniques are utilized toincrease the training set.

FIG. 10 shows three example augmentation methods used; however differentaugmentation methods can be performed. Referring to FIG. 10, 1002represents rotation of the cubes within the MPR image, rotation takesplace around the MPR centerline (see 904 of FIG. 9b ). Such a rotationmakes the network invariant to rotations around the coronary centerline,random rotations between 0 and 360 degrees around the coronarycenterline are applied to the cubes of the sequences. The secondaugmentation method (1003) makes the network invariant to slightinaccuracies in the CAD-related reference values (e.g. annotations ofthe points defining the plaque type within the section, annotation ofthe anatomical lesion severity, small errors in registration of thepullback FFR). A sequence of a section is varied by randomly choosingcenters of cubes with a stride between for instance 5±3 voxels along theMPR centerline. The third augmentation method (1004), makes the networkrobust to possible inaccuracies in the extraction of the coronary arterycenterline, the center of each cube is randomly shifted around itsorigin by for instance±2 voxels, in any direction (inside cube 1004).The result of step 708 significantly increases the number of sequencesas a result of step 707, which are used for training the network asdescribed below by step 709.

In step 709 of FIG. 7, the one or more processors implement the machinelearning network architecture and training of the network. The networkarchitecture is based on recurrent convolutional neural network (RCNN)which is employed to analyze a sequence of cubes (step 707),representing the vicinity along the extracted centerline in an MPRimage. RCNN is typically, but not limited to, built from a convolutionalneural network (CNN) with a recurrent neural network (RNN) connected inseries to analyze a sequential input.

RCNN's have been successfully used for video sequence recognition as forinstance described by Donahue et al., “Long-term recurrent convolutionalnetworks for visual recognition and description” in Proceedings of theIEEE Conference on Computer Vision and Pattern Recognition, 2015, pp.2625-2634. RCNN's are a very important variant of neural networksheavily used in language processing (Mulder et al, “A survey on theapplication of recurrent neural networks to statistical languagemodeling”, Computer Speech & Language, vol. 30, no. 1, pp. 61-98, 2015).In a general neural network, an input is processed through a number oflayers and an output is produced, with an assumption that two successiveinputs are independent of each other. This assumption is however nottrue in a number of real-life scenarios. For instance, if one wants topredict the next word in a sequence or to predict the next frame withina video sequence, it is imperative that dependence on previousobservations is considered. RCNNs are called recurrent because theyperform the same task for every element of a sequence, with the outputbeing depended on the previous computations and then the recurrent partmutually processes the previous output. Another way to think about RCNNsis that they have a “memory” which captures information about what hasbeen calculated so far. Though the input is static, the activities ofRCNN units evolve over time so that the activity of each unit ismodulated by the activities of its neighboring units. This propertyenhances the ability of the model to integrate the context information,which is important for analyzing a sequential input.

In accordance with embodiments herein, the employed RCNNs connect a CNNwith an RNN in series to analyze a sequential input. The CNN extractsimage features for each cube (or at least a portion of the cubes) of thesequence of cubes independently, and these extracted image features arethen fed to the RNN that analyzes the relevant sequential dependenciesin the whole sequence.

FIG. 11 provides an example of a RCNN network to detect coronary plaquetype and anatomical stenosis severity. The RCNN network architecturepresented in FIG. 11 should be considered as an example, and other RCNNnetwork architectures or variants can be deployed. The input of thenetwork is a sequence of cubes (1101) extracted from the MPR image(1102), along the vessel (e.g., artery) centerline (1103). Each cube isanalyzed by a three dimensional (3D) CNN (1104). The CNN includes threeconvolutional layers with kernels of 3×3×3 elements, with 32, 64, 128filters, respectively as shown by 1105. Each convolutional layer isfollowed by a 2×2×2 max-pooling (MP) layer and batch normalization whichaccelerates the training (Loffe et al, “Batch normalization:Accelerating deep network training by reducing internal covariateshift”, in Proceedings of the 32nd International Conference on MachineLearning, 2015, pp. 448-456). The features extracted by the CNN are fedto the RNN (1106). The RNN includes 2 layers of 64 Gated Recurrent Units(GRUs) each (1107). Rectified linear units (ReLU) (Glorot et al, “Deepsparse rectifier neural networks”, International Conference onArtificial Intelligence and Statistics, 2011, pp. 315-323) are used inboth CNN and RNN layers as activation functions, except for the outputlayer of the RNN. Other parameters can be used, for instance, but notlimited to, different kernel sizes and numbers, different convolutions,different dilations, different pooling, different type of recurrentunits as well as other RCNN architectures and activation functions.Furthermore, the input size of the CNN can be different as well of theuse of a 4D input voxel data for instance to support multiphase CCTAdatasets. The input image can be even a higher dimensional image, forinstance n-dimensional to support multi-energy and/or multiphase CCTAdataset. To perform both classification tasks simultaneously (plaquetype classification and anatomical lesions severity classification), forinstance but not limited to, the output of the last layer of the RNN isfed into two separate multi-class softmax classifiers (1108). Also theoutput of other RNN layers or GRU units can be used. The firstclassifier has four output units for detection of plaque type andcharacterization of its type (no-plaque, non-calcified, mixed,calcified). The second classifier has three output units for detectionof stenosis and determination of its anatomical significance(no-stenosis, non-significant stenosis, significant stenosis). Theamount of output units of the softmax classifiers can be different aswell, and will depend on the amount of classes within the CAD-relatedreference values. For instance, the anatomical lesion severity can beclassified by x-grades, then the corresponding softmax classifier equalsthe same number (x) of output units. For example, five grades; grade 0(no luminal narrowing), grade 1 (0-25% luminal narrowing), grade 2(25-50% luminal narrowing), grade 3 (50-75% luminal narrowing) grade 4(75-100% luminal narrowing).

The RCNN network of FIG. 11 is trained in a supervised manner withmini-batches and the categorical cross-entropy was used as loss functionof each softmax classifier and a L2 regularization was used for alllayers in the network. An L2 regularization is also known as leastsquares error regularization. It is basically minimizing the sum of thesquare of the differences between the target values and the estimatedvalues. The loss of the RCNN was defined as the average of the twoindividual losses (cross-entropy and L2). The described loss function isan example, other loss functions can be used as for instance mean squareerror, L1 regularization, mean bias error, mean percentage error. Eachmini-batch include of p amount of sequences of cubes (as a result fromstep 708). Typically p is around 36 sequences of cubes, but lower orhigher number of p can be used. To avoid potential bias towards the mostcommon type of plaque and stenosis in the augmented data, a stratifiedrandom data sampling was performed during the training as an optionalstep. This optional step ensures that each training iteration includestwo distinct but balanced mini-batches. One mini-batch containedsections balanced with respect to their plaque classes regardless of thestenosis significance. A second mini-batch containing sections balancedwith respect to the stenosis classes regardless of the plaque type. Incase the MPR image is constructed as a curved MPR instead of a straightMPR, as described by step 203 of FIG. 2, the RCNN network will also takethe curvature or tortuosity of the extracted centerline into account.

The network architecture described by FIG. 11 is focused on detection ofplaque type and anatomical stenosis severity. This network could also berestricted to one output, for instance only plaque type detection oronly anatomical stenosis severity classification or to multiple outputs(>2), for instance but not limited to, plaque type classification,stenosis anatomical severity classification and functionally coronarylesion severity classification or FFR value prediction. An example of anetwork architecture, which is able to classify plaque type, classifyanatomical stenosis severity and detect functionally coronary lesionseverity/predict FFR values is illustrated by FIG. 12. Within thisnetwork architecture, the same RCNN is employed as described by FIG. 11and illustrated by 121 within FIG. 12, with the addition of one outputcomponent ‘FFR Output’ (122) after the last GRU within 121. This block(122) represents the output to facilitate estimation/classification ofFFR. For estimation, of FFR values, regression could be used to estimatecontinuous FFR values between 0 and 1, this output could be, but notlimited to, the output of a linear activation function or a sigmoidactivation. For classification (discreet class assignment of FFRseverity or ranges, for instance FFR<0.7, 0.7<FFR<0.9, FFR>0.9), thisoutput could be, but not limited to, the output of a multi-sigmoidfunction (with the same as number of classes) or a softmax function(with the same number of classes).

As FFR is defined as the pressure of a distal position within a coronaryartery divided by the pressure at the proximal part of that coronaryartery, the FFR pullback will only decrease along the length of thecoronary artery, as visualized by image 504 of FIG. 5a . This knowledgecan be taken into consideration within the loss function to train theclassifier as described previously. To incorporate this decreasingbehavior of decreasing FFR along the length of the coronary artery, theloss function is adapted to allow only decreasing of FFR from proximalto distal. This can for example be achieved by looking at previous, moreproximal FFR prediction, and incorporate this information into the lossfunction.

Within an alternative embodiment, information within the myocardium istaken into consideration during assessment of the hemodynamic functionalseverity of a diseased coronary artery. As functionally significantcoronary artery stenosis causes ischemia in the myocardium (FIG. 13),which impacts the texture characteristics of the myocardium wall withina CCTA dataset, such information will enhance the assessment predictionof functionally significant coronary artery stenosis, and the predictionof the FFR along the coronary artery of interest. By way of example,embodiments may implement the method as described by Isgum et al. inU.S. Ser. No. 10/765,575B2 (Method and system for assessing vesselobstruction based on machine learning) is integrated within the RCNNnetwork architecture and illustrated by FIG. 14. Isgum et al described amethod to detect the presence of functional significant stenosis in oneor more coronary arteries based on machine learning using features ofthe myocardium, which can cope with the status of the myocardialmicrovasculature and collateral flow, without relying on the detailedmorphology of the coronary arterial system. The network as shown in FIG.14 combines the RCNN from FIG. 11 with the feature vector as describedby Isgum et al. in U.S. Ser. No. 10/765,575B2 into one new networkarchitecture. Within FIG. 14, 1401 represents the RCNN as described byFIG. 11, in which the two classifiers (1108) are removed. The featurevector from the RCNN (1401) and the feature vector as obtained from themyocardium analysis (1403) as described by U.S. Ser. No. 10/765,575B2are concatenated by block 1404. Input for the myocardium feature vector(1403) is a CCTA image dataset (1402), which is the same CCTA imagedataset in which the MPR image is created as used in the RCNN of 1401.The concatenation steps is performed by appending one feature vector(1401) to the other feature vector (1403). The results of theconcatenation step are fed into a classifier (1405) which results into aprediction of the FFR output. This classifier can be a neural network,or a support vector machine or any other supervised machine learningclassifier.

Functionally significant coronary artery stenosis causes ischemia in theventricular myocardium distal to the functionally significant coronarystenosis, as illustrated by FIG. 13. This ischemia is caused by impedesoxygen delivery to the heart muscle (myocardium) distal to the lesion,or in other words the blood supply to this myocardium region is reducedand may cause symptoms to the patient such as angina. CCTA is acquiredby applying intravenous injection in an antecubital vein, and thecontrast medium injection is timed in such a way that the coronaryarterial system contains sufficient contrast medium to clearlydistinguish the coronary artery lumen from surrounding soft tissues.This means that the injected contrast medium, once it is present in thecoronary arteries, will also be delivered to successively smallergenerations of coronary arterioles from where it traverses into thecoronary microvasculature (myocardium), which will lead to subtleenhancement of the myocardium, or reduced enhancement of the myocardiumin case of ischemia. To exploit this CCTA acquisition effect in theprediction of FFR, within an alternative embodiment the generation ofsequence of cubes as described by step 707 of FIG. 7 is adjusted toincorporate myocardium information distal to the cube of interest. Thisis further described with reference to FIG. 15.

FIG. 15 includes an image 1501 that shows a schematic view of the heartin a four chamber orientation. A short part of the coronary artery(1502) is identified and the myocardium is visualized by (1503). A smallpart of the myocardium is enlarged (1504), showing the location of thecoronary artery (1502) and the microvasculature, which provide bloodtowards the myocardium, bifurcating from the coronary artery isvisualized (1505) within the enlarged illustration of a small part ofthe myocardium. As described by step 707 of FIG. 7, the sequence ofcubes is extracted from the MPR image around a centerline point of theextracted coronary centerline (result from step 704). Image 1506illustrated the different resampling scheme. Now for each cube asextracted according to step 707, an additional cube, or cuboid, (1508)is resampled as well within the myocardium at a distance k distal to thecenterline point of interest. The size and position of the cubes ismanaged to avoid including, in the cube 1508, any foxholes within theleft or right ventricle. This can be achieved by sectioning of the bloodpool, for instance by applying thresholding techniques or other imageprocessing methods or even manually. Or alternative, by segmenting themyocardium. This can be done manually by the user or by (semi)automaticsegmentation. One example of an automatic segmentation of the LVmyocardium is given by Zreik et al “Automatic segmentation of the leftventricle in cardiac CT angiography using convolution neural networks”,2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI),2016, pp 40-43. Zreik et al, discloses a method in which the myocardiumis automatically segmented using a convolutional neural network (CNN)trained on manually annotated data.

The size of the cube (1508) or cuboid is selected to cover themyocardium, the cube may overlap towards the epicardium into theepicardial fat tissue. The results of this alternative embodiment asillustrated by FIG. 15, is that each cube has a link to anothercorresponding cube/cuboid which is located distal to the location ofinterest. This addition cube/cuboid is fed into the CNN, oralternatively new CNN's for the additional cubes/cuboids are used, asshown by the RCNN network by FIG. 12 or FIG. 14.

Once the RCNN network is trained (FIG. 11, FIG. 12, or FIG. 14), newunseen CCTA datasets are classified into the classes as defined duringthe training phase, which is further explained by the flowchart of FIG.16.

FIG. 16 illustrates a method for implementing a prediction phase, todetect plaque type, classify anatomical severity of vesselobstruction(s) and/or classify the severity of vessel obstruction(s)within unseen CCTA datasets. The unseen CCTA dataset is represented bystep 161 of FIG. 16.

Step 162 of FIG. 16 the coronary centerline is extracted which representthe center of the coronary lumen along the coronary section of interest.This step is identical of step 202 of FIG. 2.

Within step 163 of FIG. 16, a multi-planer reformatted (MPR) image iscreated along the extracted coronary section from step 162. Step 163 ofFIG. 16 is identical to step 203 of FIG. 2.

Step 164 up to and including step 168 of FIG. 16 represents block 204 ofFIG. 2 during the prediction phase; classify coronary parameters such asplaque type, anatomical coronary lesion severity and/or functionallycoronary lesion severity and/or FFR values of unseen CCTA image dataset.Prediction of the coronary parameters is performed on each centerlinepoint within the MPR image. Step 164 of FIG. 16 set the start positionto the first centerline point in the MPR image; the first position withrespect to the x-axis of the MPR image (the length of the MPR image).

Within step 165 of FIG. 16 a sequence of cubes is constructed around thecenterline position under examination. The sequence of cubes is createdin a similar matter as described by step 707 of FIG. 7 and illustratedby FIGS. 9a-9c . In step 165, a single sequence of cubes is createdaround the centerline position under examination. The size of the cubeis identical to the size as used during the training (step 707 of FIG.7) but other sizes can be used. The length of the sequence is fixed to 1cubes, and 1 is for instance 5 cubes. The distance between the cubes ism voxels, m is typically 5. Larger cubes and/or longer sequences and/ordifferent strides could be used. Special care is taken around the startand end of the MPR image, to force valid cube (cubes may partially beoutside the MPR image). To make sure that the cube contains valid MPRimage data, during resampling for example, the nearest valid voxelwithin the MPR image can be identified in case a position within thecube is outside the MPR image.

Step 166 of FIG. 16 predicts the desired coronary parameter at a currentcenterline position within the MPR image based on the sequence of cubesas a result from step 165. Within this step the same RCNN networkarchitecture is selected as used during the training phase as describedby the flowchart in FIG. 7. For instance, FIG. 11 provides an example ofa RCNN network architecture for prediction of coronary plaque type andanatomical coronary lesion severity. This RCNN is trained usingannotated plaque type and annotated anatomical lesion severity(reference value). To predict the coronary plaque type and anatomicalcoronary lesion severity on unseen image data, step 166 uses the sameRCNN network architecture and the trained model of this RCNNarchitecture is used. FIG. 12 provides a RCNN network architecture toassess plaque type, anatomical lesion severity and the functionallycoronary severity and is trained on the annotated plaque type, annotatedanatomical lesion severity and the pullback data obtained from invasiveFFR measurements. During prediction, the same RCNN architecture is usedas illustrated by FIG. 12 and the trained model of this RCNNarchitecture is selected. The same is true for the RCNN architectureprovided by FIG. 14, in which only the functionally lesion severity istrained and predicted by the current step.

Step 167 of FIG. 16 sets the centerline position under examination to anew location. This can be the next centerline position within theextracted coronary centerline tree (i.e. next location within the MPRimage along the length of the MPR image), or a centerline position witha predefined distance with respect to current centerline location.

Step 165, 166 and 167 are repeated as illustrated by block 168 until thelast centerline position within the extracted coronary centerline isreached.

Finally step 169 provides several methods to present the results and isidentical to step 206 of FIG. 2.

Within an alternative embodiment, the machine learning algorithmdeployed to assess the hemodynamic functional severity of a coronaryartery within block 204 of FIG. 2 uses a different network architecture,and results in a slightly different flowchart for both the learningphase as well for the prediction phase. FIG. 17 illustrates the overallworkflow. Within a CCTA scan (1701), the centerlines of the coronaryarteries are extracted and used to reconstruct multi-planar reformattedimages of the coronary arteries (1702). Then, an unsuperviseddimensionality reduction step is applied, where the MPR image (volume)of a complete artery is compressed into a fixed number of features(encodings) using two unsupervised autoencoders, applied sequentially:first) a 3D variational convolutional autoencoder 1703 (3D-VCAE), thatspatially compresses local sub-volumes (cubes) of the coronary artery,and second) a 1D convolutional autoencoder 1704 (1D-CAE), thatsequentially compresses the encodings of the complete coronary arteryobtained previously by the 3D-VCAE. Then the final extracted encodingsare employed in a supervised fashion with a support vector machine (SVM)classifier or any other supervised classifier 1705, to classify arteries(and patients) according to FFR.

FIG. 18 shows the detailed flowchart to be employed during the trainingphase to train the vessel obstruction assessment model. In step 1801 ofFIG. 18 the reference standard database is obtained and used to trainthe machine learning network. The reference standard database is adatabase which contains data of multiple patients. For example, thedatabase contains, for each patent, a) contrast enhanced imaging (e.g.CT) datasets (step 1803) and corresponding b) invasively measuredfractional flow reserve reference values (step 1802). The fractionalflow reserve reference values correspond to pressure measurements thatare invasively measured during a pressure wire pullback operation or ata single point proximate the distal position of the coronary vessel ofinterest. Other hemodynamic indices can be used to assess functionallysignificant coronary artery obstruction(s) such as coronary flowreserve, instantaneous wave-free ratio, hyperemic myocardium perfusion,index of microcirculatory resistance and pressure drop along a coronaryartery.

In step 1804 of FIG. 18, the processors extract the axial trajectory(e.g., coronary centerline) which represent the center of the vessel ofinterest (VOI) lumen along the VOI section of interest. This step issubstantially similar to step 202 of FIG. 2.

In step 1805 of FIG. 18, the processors create the MPR image along theextracted coronary centerline as a result from step 1804. Step 1805 ofFIG. 18 is identical to step 203 of FIG. 2.

In step 1806 of FIG. 18, the processors align the FFR reference value1802 to the spatial coordinates of the MPR image as a result of step1805. This step is substantially similar to step 706 of FIG. 7. In casethe FFR reference values are obtained by using the MPR image as aresults from step 1805, this step is skipped. This step is also skippedin case the FFR reference value consist a single point value.

In step 1807 of FIG. 18, the processors generate the sequence of cuboidsfrom the MPR image which are used to train the machine learning basedvessel obstruction assessment model as described further within theremaining step of current workflow. The generation of the sequence ofcuboids is illustrated by FIGS. 19a-19c . Along the MPR image (1901 ofFIG. 19a ) a sequence of cuboids (image 1902 of FIG. 19b ) is createdwhich resamples the MPR image into smaller volumes. The size of thecuboid is defined that it contains the whole arterial lumen and a partwithin the vicinity of the artery. Each cuboid is typically 40×40×5voxels, but other cuboid sizes are possible. The distance between thecuboids is m centerline points (m is typically 1 voxel, but otherdistances between the cuboids can be possible). In case that thedistance between the centerline points is one voxel, the length of thesequence of cuboids is identical to the length of the extractedcenterline and to the length of the MPR image.

In step 1808 of FIG. 18, the processors extract features from thecuboids. In step 1809 of FIG. 18 the processors extract features fromthe extracted cuboid features. For example, the features characterizingthe pressure drop within a VOI (e.g. coronary artery of interest) areextracted from the MPR image by analyzing the sequence of cuboids as aresult from step 1807. As blood flow and pressure in a coronary arterymay be affected by several coronary artery stenoses located along theartery at hand. Different combinations of stenoses locations and grades,i.e. degrees of lumen narrowing, may cause a drop in FFR along theartery. Therefore, to detect drops in FFR within an artery, localanalysis around the stenoses may be insufficient, and instead volumetricanalysis of the complete artery should be performed. Coronary arteriesare complex anatomical 3D structures, with varying lengths and anomaliesacross patients. The resolution of modern CT scanners is high and alarge number of voxels (millions) is contained in a volumetric imagedata set for a single artery and following the straightforward approachof training a single convolutional autoencoder (CAE), applied directlyto the complete (or a substantial majority of) artery volume without alarge reconstruction error, is unlikely. To address the foregoingconcerns, the CAE network architecture used in accordance withembodiments herein includes a two-stage encoding approach to encode acomplete MPR image (volume) of the coronary artery, regardless of itslength and anatomy.

A CAE compresses the data from an input image to a small vector thatcontains enough information to reconstruct the input image by thedecoder. By this the autoencoder is forced to learn features about theimage being compressed. This is illustrated by FIGS. 19a-19c . Image1901 of FIG. 19a shows an example of the extracted MPR image as a resultfrom step 203 of FIG. 2 and further illustrated by FIGS. 4a-4e . Theproximal part which is at the start of the centerline which identifiesthe coronary of interest is at the left side of image 1901 and the endof centerline is at the right side of the MPR image. Image 1902 of FIG.19b illustrates several features (information) along the length of theMPR image extracted by the encoder of a CAE. Image 1903 of FIG. 19cshows the results of the decoder, reconstructing the original image. Atypical CAE includes two major parts, an encoder and a decoder. Theencoder compresses (encodes) the data to lower dimensional latent spaceby convolutional operations and down-sampling (max-pooling), andsubsequently expands (decodes) the compressed form to reconstruct theinput data by deconvolutional operations and upsampling (unpooling).Training the CAE, while minimizing a distance loss between the encoderinput and the decoder output, ensures that the abstract encodings,generated from the input, contain sufficient information to reconstructit with low error. Once the CAE is trained, the decoder is removed, andthe encoder is used to generate encodings for unseen data.

FIG. 20 illustrates the encoding flow. First, represented by step 1808,a 3D variational convolutional autoencoder (3D-VCAE) is applied (2003)on local sub-volume (cuboid 2002) extracted from the MPR image 2001along the artery centerline (as in FIG. 18, 1807). The 3D-VCAE encodeseach sub-volume into a set of encodings. When applied to all sequentialsub-volumes (within a step size of 1) along the extracted coronarycenterline (1804 of FIG. 18), the result is a 2D features map of thesame length as the artery. This 2D features map is then represented as aset of 1D sequences of encodings, running along the artery (2004). In apreferred embodiment, the features characterizing the pressure dropwithin a coronary artery of interest are characterized by the featuresin an unsupervised manner, extracted by a 3D-VCAE applied on the localsub-volumes (cuboids). Alternative, any other engineered characteristicthat describes the pressure drop along a coronary artery by the sequenceof cuboids can be employed (e.g. Gaussian filters, Haralick texturefeatures) and/or morphology (e.g. coronary lumen, coronary volume, lumenintensity) can be used as features. An example of such alternativeengineered feature method designed to quantify the perceived texturewithin the cuboids is by computing Haralick texture features, whichcaptures numerical features of a texture using spatial relations ofsimilar gray tones (Robert M. Haralick et al., “Textural Features forImage Classification”, IEEE Transactions on Systems, Man, andCybernetics, 1973, SMC-3 (6): 610-621). Any combination of thesefeatures can be selected.

Finally, represented by step 1809 of FIG. 18, the processors apply a 1Dconvolutional autoencoder (1D-CAE) to each sequence of encodingsseparately (2005 of FIG. 20). Hence, the 1D-CAE encodes the varyinglength sequences of encodings further into a fixed number of encodingsthat represent the complete artery, regardless of its length.

3D-VCAE

Variational autoencoder (VAE) are generative models, which approximatedata generating distributions as described by Kingma et al,“Auto-encoding variational bayes”, arXiv preprint arXiv:1312.6114, 2013.Through approximation and compression, the resulting models capture theunderlying data manifold; a constrained, smooth, continuous, lowerdimensional latent (feature) space where data is distributed (Kingma etal, “Semi-supervised learning with deep generative models”, Advances inneural information processing systems, 2014, pp. 3581-3589). Theseadvantageous properties of the latent space is used within currentembodiment to employ a VCAE to compress and encode local volumes alongthe artery. To capture local volumetric characteristics of the artery,the input to the 3D-VCAE is set to a volume as described by step 1807.The output of the encoder in the 3D-VCAE is set to a predefined numberof encodings (2004), for instance 16. To encode the complete artery,overlapping volumes with stride of 1 voxel are extracted and encodedwith 3D-VCAE. This results, in case of 16 numbers of encodings percenterline points, in 16×L encodings, where L is the length of theextracted coronary centerline. The detailed 3D-VCAE architecture isshown in FIG. 21a . FIG. 21a shows the detailed network architecture ofthe 3D-VCAE. The input and outputs are volumes of for instance 40×40×5voxels. Within FIG. 20a , the key: N @ (sizekernel) is a convolutionallayer with N kernels of size sizekernel. MP @ (sizekernel) is amax-pooling layer with kernel size sizekernel. US @ (sizekernel) is anupsampling layer with kernel size sizekernel. FC @ (Nunits) is a fullyconnected layer with Nunits units. Once the 3D-VCAE is trained, theoutput of the μ layer is used to generate encodings for the input.Descending and ascending arrows represent the encoder and the decoder ineach autoencoder, respectively. In the 3D-VCAE, batch normalizationlayers and rectified linear units (ReLUs) are used after all convolutionlayers except the encoder and decoder output layers. Other parameterscan be used, for instance different kernel sizes, number of encodinglayers as well as other CAE architectures and activation functions.Furthermore, the input size of the CAE can be different as well of theuse of a 4D voxel patch for instance to support multiphase CCTAdatasets. The input patch can be even a higher dimensional voxel patch,for instance n-dimensional to support multi-energy and/or multiphaseCCTA dataset.

1D-CAE

When representing the coronary artery to predict FFR, characteristicsalong the artery, starting from the ostium to the most distal part ofthe extracted coronary centerline, are taken into account. Therefore,the local encodings from the 3D-VCAE are further compressed by the1D-CAE 2005 while analyzing the complete artery at once. To accomplishthis, the 2D features map, includes a set of the for example 16encodings generated by 3D-VCAE at each coronary artery center point, isrepresented as a set of 1D sequences 2004. Each sequence includes 1×Lencodings, where L represents the length of the artery, i.e. number ofcoronary artery center points. This representation leads each sequenceto represent a specific encoding along the artery (2004), andconsequently, allows to apply a simple 1D-CAE to each of the 16sequences of encodings separately (2005). The weights of the 16 1D-CAEsare shared, where each 1D-CAE encodes one of the 16 sequences into forexample 64 encodings. This results in n features (2006) that representthe complete coronary artery of interest. In our example n is 16×64,1024 features. The detailed 1D-CAE architecture is shown in FIG. 21b .The input and outputs are for instance 16×L sequences of encodings.Within FIG. 21b , the key: N @ (sizekernel:sizestride) is a 1Dconvolutional layer with N kernels of size sizekernel and stride ofsizestride. US @ (sizekernel) is a 1D upsampling layer with kernel sizesizekernel. The 1D-CAE is applied separately, but with shared weights,on each one of the 16 1D-sequences. Once the 1D-CAE is trained, theoutput of the e layer (FIG. 21b ) is used to generate encodings for eachinput sequence. Descending and ascending arrows represent the encoderand the decoder in each autoencoder, respectively. In the 1D-CAE, theexponential linear units (ELUs) are used after all convolutions layersexcept the encoder and decoder output layers. Other parameters can beused, for instance different kernel sizes, number of encoding layers aswell as other CAE architectures and activation functions.

Within step 1810 of FIG. 18, the processors apply a supervisedclassifier to train an FFR classifier. Several supervised classifierscould be used, for instance, a support vector machine (SVM) classifier.An SVM is a supervised machine learning classifier that can be employedfor both classification and regression purposes. SVMs are based on theidea of finding a hyperplane (221, FIG. 22) that best divides a datasetinto predefined classes (222, FIG. 22). As a simple example, for aclassification task with only two features is illustrated in FIG. 22.During training of the SVM, a hyperplane that best separates samples oftwo classes is found by maximizing the margin around the decisionboundary while minimizing the number of training samples within themargin (FIG. 22). The margin (223, FIG. 22) is determined by the supportvectors (224, FIG. 22) i.e. training samples that lie on the margin.Intuitively, a good separation is achieved by the hyperplane that hasthe largest distance to the nearest training-data point of any class. Inother words, the distance between the hyperplane and the nearest supportvector from either set is known as the margin. The goal of SVM is tofind a hyperplane with the greatest possible margin between thehyperplane and any point (support vector) within the training set. Otherkinds of classifiers may include neural networks, Bayesian classifiers,Tree Ensembles (e.g., random Forests) (Kotsiantis et al, “SupervisedMachine Learning: A Review of Classification Techniques”, Informatica31, 2007, 249-268). To be able to use a supervised (SVM) classifier,reference data must be present that can be used as a reference standard.The reference standard is a database from multiple patients (step 1801).Each set within this database contains a) contrast enhanced CT datasets(step 1803) with belonging b) reference value (step 1802). In apreferred embodiment, the reference value indicative for functionalsignificance coronary lesion (e.g. FFR) 1802, representing afluid-dynamic parameter, is an (pullback) invasive fractional flowreserve (FFR) measurement as performed during X-ray angiography whichbelongs to the contrast enhanced CT dataset 1803. For example, FFR ismeasured with a coronary pressure guidewire at maximal hyperemia inducedby intravenous adenosine. During X-ray angiography the FFR wire isplaced as distally as possible in the target vessel and FFR is assessedby means of a manual or automatic pullback in the distal part of thetarget vessel. Finally, the FFR wire is retrieved at the level of theguiding catheter to achieve an FFR value of 1.00 in order to assess thequality of the measurement performed. When multiple FFR measurements areavailable due to repeated measurements or multiple stenosis, the minimalvalue is taken as the standard of reference. The FFR reference value1802 can be any parameter which links the patient specific CCTA datasetsto myocardial ischemia of that patient. For instance, the referencevalue (1802) can be the measured coronary flow reserve or the index ofmicrocirculatory resistance which provides a measurement of the minimumachievable microcirculatory resistance in a target coronary arteryterritory, enabling a quantitative assessment of the microvascularintegrity. Other examples of different parameters for the referencevalue (1802) are the occurrence of major adverse cardiac events (MACE)within a predefined amount of time after acquisition of the CCTAdataset, or if the patient underwent revascularization within apredefined amount of time after acquisition of the CCTA dataset, or theresults of cardiac stress test, the results of myocardial magneticresonance imaging (MRI) perfusion, SPECT, PET, CT perfusion, orultrasound.

Using a database of FFR reference values 1802, which corresponds to theused CCTA dataset 1803, each reference value (1802) is marked asbelonging to one of two classes, for instance “functionally significantstenosis present” (invasive FFR<for instance 0.8) or “no significantstenosis present” (invasive FFR>for instance 0.8) (the known labels),the SVM classifier learns to separate the different classes. First, eachtraining sample (e.g. CCTA dataset) is represented as a point in ann-dimensional feature space, where n is the number of computed features(e.g. the number of features in the feature vector, the result of step1809 of FIG. 18). For all reaming CCTA cases in the database 1803, sucha feature vector is computed. All training samples (e.g. CCTA cases inthe database) are used to train the chosen classifier. In the trainingphase, the SVM classifier finds the hyperplane that makes the bestseparation between the classes i.e. by finding a hyperplane separatingtwo classes with the largest margin as illustrated in FIG. 22.

SVM is in nature a two-class classifier. Nevertheless, multi-classclassification, i.e. classification in multiple classes, can beperformed by e.g. performing multiple 2-class classifications (e.g.chosen class vs. all remaining classes or between every pair ofclasses—one vs one). Hence, the FFR classifier (1810, FIG. 18) can betrained to recognize multiple classes, for example “no functionallysignificant stenosis present”, “mild functionally significant stenosispresent” or “severe functionally significant stenosis present”, or anycategories chosen based on the reference value (step 1802 of FIG. 18).When the reference value (FIG. 18, 1802) is an invasive FFR measurement,above classification can be achieved using for instance the followinginvasive FFR threshold values:

-   -   i) Invasive FFR>0.9—“no functionally significant stenosis        present”    -   ii) Invasive FFR between 0.7 and 0.8—“mild functionally        significant stenosis present”    -   iii) Invasive FFR<0.7—“severe functionally significant stenosis        present”

Once the system is trained, new unseen CCTA datasets are classified intothe classes as defined during the training phase, which is furtherexplained by the flowchart of FIG. 23. FIG. 23 illustrates a frameworkfor implementing the prediction phase, to classify the severity ofvessel obstruction(s), or detect the FFR value within unseen CCTAdatasets; to determine the presents of functionally significant stenosisin one or more coronary arteries from a CCTA dataset. The unseen CCTAdataset is represented by step 2301 of FIG. 23.

Step 2302 of FIG. 23 the coronary centerline is extracted whichrepresent the center of the coronary lumen along the coronary section ofinterest. This step is identical of step 202 of FIG. 2.

Step 2303 of FIG. 23 a multi-planer reformatted (MPR) image is createdalong the extracted coronary section from step 2302. Step 2303 of FIG.23 is identical to step 203 of FIG. 2.

Step 2304 up to and including step 2307 of FIG. 23 represents block 204of FIG. 2 during the prediction phase and classifies the functionallycoronary lesion severity of unseen CCTA image dataset. Prediction of thefunctionally coronary lesion severity is performed on the completecoronary of interest represented within the MPR image. Within step 2304of FIG. 23 the generation of the sequence of cuboids from the MPR imageis performed and is identical to step 1607 of FIG. 16.

Within step 2305 and 2306 of FIG. 23 the computing of the features takesplace and is identical as described previously for step 1808 and 1809 ofFIG. 18.

In step 2307 of FIG. 23, the FFR classifier assigns new unseen CCTAdatasets into the categories as defined during the training phase. Thisclassifier is the same classifier as used in block 1810 in FIG. 18.Within the prediction phase, unseen CCTA dataset are mapped by step 2307of FIG. 23 in the n-dimensional feature space and its location in thisfeature space with respect to the hyperplane determines its class label.This classification results in an assessment of the severity that one ormore coronary obstructions impedes oxygen delivery to the heart muscleand is represented by step 2308 of FIG. 23. FIG. 24 shows a visualrepresentation of the classifier. The result of step 2306 of FIG. 23,representing the feature vector of the unseen image, is the input (241)of the classifier. Label 242 of FIG. 24 represents the FFRclassification model, as learned during the learning phase as describedby step 1810 of FIG. 18. Label 243 of FIG. 24 represent the output ofthe classifier (FIG. 23, 2308) incase two classes are learned; positivemeaning one or more functionally significant coronary lesions present,and negative meaning no functionally significant coronary lesionpresent.

Finally step 2308 provides the output and present the results and isidentical to step 206 of FIG. 2.

Within an alternative embodiment, the machine learning algorithmrepresented by block 204 of FIG. 2 is deployed to assess the hemodynamicfunctional severity of one or more coronary vessel obstructions of atarget organ, being the heart, based on contrast enhanced volumetricimage dataset. FIG. 25 represent the flowchart of the training phase ofthe machine learning algorithm within this alternative embodiment andcombines the method described by FIGS. 19a-19c with the teaching fromIsgum et al. in U.S. Ser. No. 10/765,575B2 to assess the hemodynamicfunctional severity of one or more coronary vessel obstructions within acontrast enhanced volumetric image dataset. Features, of all extractedcoronary arteries, with centerlines extracted like in (1804 FIG. 18),are extracted in an identical way as described by step 1801 up to andincluding step 1809 of FIG. 18. For instance, n coronary features (2006)are extracted (for instance 1024 features) per MPR image 2505 (percoronary artery) as described by FIG. 20. Then these features 2509 arecombined with the feature vector (for instance a feature vector of 512myocardium features) as obtained from the myocardium analysis (2512) asdescribed by U.S. Ser. No. 10/765,575B2 by concatenation 2511. Input forthe myocardium feature vector (2512) is a CCTA image dataset (2503). Theconcatenation steps is performed by appending the feature vectorresulting from the coronary analysis (2509) to the feature vector 1412resulting from the myocardium analysis. The results of the concatenationstep are used to train a multi instance learning (MTh) FFR classifier2513 as for instance described by Ilse et al in “Attention-based deepmultiple instance learning”, arXiv preprint arXiv:1802.04712. To be ableto use such MTh classifier, each patient is represented as a bag ofinstances that includes all coronaries (with the extracted features) anda myocardium (with its extracted features). To build the bag ofinstances, the features of the coronaries are concatenated element wisewith the features of the myocardium: features of each coronary i (2509)is concatenated (2511) with the myocardium features (2512), for i from 1to N (total amount of coronaries represented by 2510). The result is amatrix with N rows and total number of features per row (e.g. 1536features exciting of 512 for myocardium and 1024 for each coronary). Thereference value 2502 belonging to an image dataset 2503 could be set,but not limited to, to the minimal invasively measured FFR in thatpatient (image dataset).

FIG. 26 represent the flowchart during the prediction phase of themachine learning algorithm within this alternative embodiment in anassessment of the severity that one or more coronary obstructionsimpedes oxygen delivery to the heart muscle from a CCTA dataset. Theunseen CCTA dataset is represented by step 2601 of FIG. 26.

Step 2602 of FIG. 26 the coronary centerline(s) are extracted whichrepresent the center of the coronary lumen along each coronary sectionof interest. This step may be implemented in a manner substantiallysimilar to step 202 of FIG. 2.

Step 2603 up to and including step 2606 of FIG. 26 are identical to step2303 up to and including step 2306 of FIG. 23.

Step 2608 of FIG. 26 concatenate the feature vectors obtained from theunseen CCTA dataset based on coronary analysis 2606 and myocardiumanalysis 2609 in a similar way as described by block 2511 in FIG. 25 andincluding all available coronary centerlines (2607). The result of thisconcatenation step is fed into the MIL FFR classifier 2610 which hasbeen trained as described by step 2513 of FIG. 25. Finally step 2611provides the classification results from step 2610 to the user and maybe implemented in a manner substantially similar to step 206 of FIG. 2.

With respect to step 203 of FIG. 2 which describes the creation of theMPR image, the current specification is mainly focus on the use of astraight MPR image. The description of step 203 includes the use of acurved MPR image, which allows curvature or tortuosity information ofthe extracted centerline can be taken into account within the describedmachine learning network architectures. Alternatively, this step can beskipped, and the generation of cubes as described by the relevantflowcharts can be performed directly on the image dataset using theextracted centerline as a result from step 202 of FIG. 2. In this way,curvature or tortuosity information is implicitly taken into accountduring training the machine learning based VAO models and also duringthe prediction phase.

With respect to prediction of the functionally significance of acoronary artery, either along the extracted coronary centerline or perextracted coronary centerline, it can be challenging to obtain asufficient amount of invasively measured hemodynamic reference values.These invasively measured hemodynamic FFR reference values can beinvasively measure FFR at the distal location of the coronary artery orinvasively measure pullback FFR in case of prediction of thefunctionally significance of a coronary artery along the extractedcoronary centerline.

As mentioned before, the reference standard (701, 1801, 2501) exist aspart of a database which contains for each patent a) contrast enhancedCT datasets (703, 1803, 2503) and corresponding b) FFR reference values(702, 1802, 2502) such as for instance invasively measured fractionalflow reserve. In most of the cases also x-ray angiographic image data isavailable of the patient. Within an alternative embodiment the FFRreference value (702 of FIG. 7 or 1802 of FIG. 18 or 2502 of FIG. 25)are obtained by calculation of the FFR pullback or calculation of thedistal FFR value eliminating the need for invasively measure hemodynamicparameters such as for instance FFR. This can be performed by using thex-ray angiographic image data of the patient and compute the (pullback)FFR value by using for instance the vFFR (vessel-FFR) workflow withinCAAS Workstation 8.0 (Pie Medical Imaging, the Netherlands). Due to thehigh spatial resolution of X-ray angiography and the method employedwithin CAAS Workstation vFFR workflow, the accuracy of the computed vFFRis considerably high, as presented at the EuroPCR 2018 by Masdjedi etal, “Validation of 3-Dimensional Quantitative Coronary Angiography basedsoftware to calculate vessel-FFR (the FAST study)”.

The vFFR method of CAAS Workstation generates a 3D coronaryreconstruction using 2 angiographic x-ray projections with at least 30degrees apart. vFFR is calculated instantaneously by utilizing aproprietary algorithm which incorporates the morphology of the 3Dcoronary reconstruction and routinely measured patient specific aorticpressure. FIG. 27 shows an example of obtaining a computed FFR pullbackof the coronary circumflex by using CAAS Workstation. 2701 shows thesegmentation of the coronary circumflex in each 2D X-ray angiographicimage, resulting in a 3D reconstruction of the coronary artery (2702).The graph 2703 shows the computed vFFR value along the length of the 3Dreconstructed coronary artery. The same approach could also be performedon the CCTA dataset, eliminating the need to corresponding x-rayangiographic image data for each patient.

The present disclosure mainly describes the organ of interest as themyocardium and the vessels being the coronary arteries. The skilledperson would appreciate that this teaching can be equally extended toother organs. For instance, the organ of interest can be the kidney,which is perfused by the renal arteries, or (parts) of the brain asperfused by the intracranial arteries. Furthermore, the presentdisclosure refers to CCTA datasets (in several forms). The skilledperson would appreciate that this teaching can be equally extended toother imaging modalities, for instance rotational angiography, MRI,SPECT, PET, Ultrasound, X-ray, or the like.

The embodiment of this disclosure can be used on a standalone system orincluded directly in, for instance, a computed tomography (CT) system.FIG. 28 illustrates an example of a high-level block diagram of acomputed tomography (CT) system. In this block diagram the embodiment isincluded as an example how the present embodiment could integrate insuch a system.

Portions of the system (as defined by various functional blocks) may beimplemented with dedicated hardware, analog and/or digital circuitry,and/or one or more processors operating program instructions stored inmemory.

The most common form of computed tomography is X-ray CT, but many othertypes of CT exist, such as dual-energy, spectral, multi-energy, orphoton-counting CT. Also positron emission tomography (PET) andsingle-photon emission computed tomography (SPECT) or combined with anyprevious form of CT.

The CT system of FIG. 28 describes an X-ray CT system. In an X-ray CTsystem an X-ray system moves around a patient in a gantry and obtainsimages. Through use of digital processing a three-dimensional image isconstructed from a large series of two-dimensional angiographic imagestaken around a single axis of rotation.

For a typical X-ray CT system 120 an operator positions a patient 1200on the patient table 1201 and provides input for the scan using anoperating console 1202. The operating console 1202 typically comprisesof a computer, a keyboard/foot paddle/touchscreen and one or multiplemonitors.

An operational control computer 1203 uses the operator console input toinstruct the gantry 1204 to rotate but also sends instructions to thepatient table 1201 and the X-ray system 1205 to perform a scan.

Using a selected scanning protocol selected in the operator console1202, the operational control computer 1203 sends a series of commandsto the gantry 1204, the patient table 1201 and the X-ray system 1205.The gantry 1204 then reaches and maintains a constant rotational speedduring the entire scan. The patient table 1201 reaches the desiredstarting location and maintains a constant speed during the entire scanprocess.

The X-ray system 1205 includes an X-ray tube 1206 with a high voltagegenerator 1207 that generates an X-ray beam 1208.

The high voltage generator 1207 controls and delivers power to the X-raytube 1206. The high voltage generator 1207 applies a high voltage acrossthe vacuum gap between the cathode and the rotating anode of the X-raytube 1206.

Due to the voltage applied to the X-ray tube 1206, electron transferoccurs from the cathode to the anode of the X-ray tube 1206 resulting inX-ray photon generating effect also called Bremsstrahlung. The generatedphotons form an X-ray beam 1208 directed to the image detector 1209.

An X-ray beam 1208 comprises of photons with a spectrum of energies thatrange up to a maximum determined by among others the voltage and currentsubmitted to the X-ray tube 1206.

The X-ray beam 1208 then passes through the patient 1200 that lies on amoving table 1201. The X-ray photons of the X-ray beam 1208 penetratethe tissue of the patient to a varying degree. Different structures inthe patient 1200 absorb different fractions of the radiation, modulatingthe beam intensity.

The modulated X-ray beam 1208′ that exits from the patient 1200 isdetected by the image detector 1209 that is located opposite of theX-ray tube.

This image detector 1209 can either be an indirect or a direct detectionsystem.

In case of an indirect detection system, the image detector 1209comprises of a vacuum tube (the X-ray image intensifier) that convertsthe X-ray exit beam

1208′ into an amplified visible light image. This amplified visiblelight image is then transmitted to a visible light image receptor suchas a digital video camera for image display and recording. This resultsin a digital image signal.

In case of a direct detection system, the image detector 1209 comprisesof a flat panel detector. The flat panel detector directly converts theX-ray exit beam 1208′ into a digital image signal.

The digital image signal resulting from the image detector 1209 ispassed to the image generator 1210 for processing. Typically, the imagegeneration system contains high-speed computers and digital signalprocessing chips. The acquired data are preprocessed and enhanced beforethey are sent to the display device 1202 for operator viewing and to thedata storage device 1211 for archiving.

In the gantry the X-ray system is positioned in such a manner that thepatient 1200 and the moving table 1201 lie between the X-ray tube 1206and the image detector 1209.

In contrast enhanced CT scans, the injection of contrast agent must besynchronized with the scan. The contrast injector 1212 is controlled bythe operational control computer 1203.

For FFR measurements an FFR guidewire 1213 is present, also adenosine isinjected by an injector 1214 into the patient to induce a state ofmaximal hyperemia.

An embodiment of the present application is implemented by the X-ray CTsystem 120 of FIG. 18 as follows. A clinician or other user acquires aCT scan of a patient 1200 by selecting a scanning protocol using theoperator console 1202. The patient 1200 lies on the adjustable table1201 that moves at a continuous speed during the entire scan controlledby the operational control computer 1203. The gantry 1204 maintains aconstant rotational speed during the entire scan

Multiple two-dimensional X-ray images are then generated using the highvoltage generator 1207, the X-ray tube 1206, the image detector 1209 andthe digital image generator 1210 as described above. This image is thenstored on the hard drive 1211. Using these X-ray images, athree-dimensional image is constructed by the image generator 1210.

The general processing unit 1215 uses the three-dimensional image toperform the classification as described above.

There have been described and illustrated herein several embodiments ofa method and apparatus for automatically identify patients withfunctionally significant stenosis, based on the information extractedfrom a single CCTA image only.

Hereafter, a summary is provided of various aspects of embodimentsherein that may be claimed alone or in any combination thereof.

Example 1

A method for assessing a vessel obstruction, the method comprising:

obtaining a volumetric image dataset for a target organ that includes avessel of interest;

extracting an axial trajectory extending along of a vessel of interest(VOI) within the volumetric image dataset;

creating a three-dimensional (3D) multi-planer reformatted (MPR) imagebased on the volumetric image dataset and the axial trajectory of theVOI; and

extracting a VOI parameter from the MPR image utilizing a machinelearning-based vessel obstruction assessment (VOA) model.

Example 2

The method of one or more of the examples herein, further comprisingimplementing a prediction phase to at least one of i) detect plaquetype; ii) classifying anatomical severity of vessel blockage; and/oriii) classifying a hemodynamic severity of vessel obstructions within anunseen portion of the volumetric image data set.

Example 3

The method of one or more of the examples herein, wherein the machinelearning-based VOA model generates a sequence of cubes from the MPRimage, each of the cubes including a group of voxel from the MPR image,the sequence of cubes created within sections of the VOI resulting in asequence of cubes for corresponding sections.

Example 4

The method of one or more of the examples herein, wherein the machinelearning-based VOA model extracts image features associated with cubesfrom the sequence of cubes independently.

Example 5

The method of one or more of the examples herein, wherein the machinelearning analyzes the image features in sequential dependence.

Example 6

The method of one or more of the examples herein, wherein a size of thecube is defined to contain a whole lumen for the VOI and a portion oftissue outside of the lumen to facilitate extracting the VOI parameterin connection with positive remodeling, wherein positive remodelingrefers to a direction of atherosclerotic plaque growth.

Example 7

The method of one or more of the examples herein, wherein the axialtrajectory corresponds to a coronary centerline of the VOI, the coronarycenterline representing a center of a coronary lumen along a coronarysection of interest, the axial trajectory may correspond to a singlecoronary artery, a coronary bifurcation or a full coronary tree,wherein, when the coronary section of interest includes one or morebifurcation(s), the coronary centerline includes the one or morebifurcations.

Example 8

The method of one or more of the examples herein, wherein the machinelearning-based VOA model is based on a recurrent convolutional neuralnetwork (RCNN) which is employed to analyze a vicinity along the axialtrajectory of the VOI in the MPR image, the RCNN connects aconvolutional neural network (CNN) with a recurrent neural network (RNN)connected in series to analyze the portion of the MPR along the axialtrajectory as a sequential input.

Example 9

The method of one or more of the examples herein, wherein the machinelearning-based VOA model applies at least one convolution layer followedby a max pooling layer to extract an image feature of interest from theMPR image and utilizes classifiers for at least one of detecting plaquetype, characterizing plaque type, detecting stenosis or determining ananatomical significance of a stenosis.

Example 10

The method of one or more of the examples herein, wherein the machinelearning-based VOA model includes a feature extraction for creating afeature vector based on the MPR image, the feature vector comprises aseries of factors that are measured or extracted from a referencedatabase of images, the series of factors describing or characterizingthe nature of a corresponding wall region of the vessel of interest, themachine learning-based VOA model further including a classifier toclassify the feature vector extracted from the MPR image.

Example 11

The method of one or more of the examples herein, wherein the VOIparameter includes at least one of coronary plaque type, anatomicalcoronary lesion severity or functionally significant coronary lesionseverity, and wherein the machine learning-based VOA model assesses atleast one of i) functionally significant coronary lesion severity, ii)plaque type or iii) anatomical coronary lesion severity.

Example 12

A method to train a vessel obstruction assessment (VOA) model,comprising:

obtaining a training database that includes volumetric imaging datasetsfor multiple patients and corresponding coronary artery disease (CAD)related reference values, the volumetric image data sets being for atarget organ that includes a vessel of interest, the CAD relatedreference values corresponding to one or more points along a vessel ofinterest within the corresponding imaging data set; and

for at least a portion of the volumetric image data sets andcorresponding CAD related reference values,

-   -   extracting an axial trajectory extending along of a vessel of        interest (VOI) within the corresponding volumetric image        dataset,    -   creating a three-dimensional (3D) multi-planer reformatted (MPR)        image based on the corresponding volumetric image dataset and        the axial trajectory of the VOI, the MPR image extending along        the axial trajectory of the VOI, and    -   training a machine learning-based vessel obstruction assessment        (VOA) model based on the MPR images, the training further        comprising extracting, from the MPR images, features        characterizing a CAD related parameter along the axial        trajectory within the VOI.

Example 13

The method of one or more of the examples herein, further comprisingaligning the CAD related reference values to spatial coordinates of thecorresponding MPR images.

Example 14

The method of one or more of the examples herein, further comprisinggenerating a sequence of cubes from the corresponding MPR images, eachof the cubes including a group of voxel from the corresponding MPRimage, the sequence of cubes created within sections of the VOIresulting in a sequence of cubes for corresponding sections.

Example 15

The method of one or more of the examples herein, wherein the trainingfurther comprises applying a convolutional neural network to a sequenceof cubes alone the MPR image to build the machine learning-based VOAmodel.

Example 16

The method of one or more of the examples herein, wherein the applyingfurther comprises generating a set of encodings at points along theaxial trajectory to form a set of one-dimensional (1D) sequences, eachof the 1D sequences representing a specific encoding along the VOI, thetraining further comprising applying a supervised classifier to learn afractional flow reserve (FFR) classifier based on the 1D sequences.

Example 17

A system for assessing a vessel obstruction, comprising:

memory configured to store a volumetric image dataset for a target organthat includes a vessel of interest; and

one or more processors that, when executing program instructions storedin the memory, are configured to:

-   -   extract an axial trajectory extending along of a vessel of        interest (VOI) within the volumetric image dataset,    -   create a three-dimensional (3D) multi-planer reformatted (MPR)        image based on the volumetric image dataset and the axial        trajectory of the VOI, and    -   extract a VOI parameter from the MPR image utilizing a machine        learning-based vessel obstruction assessment (VOA) model.

Example 18

The system of one or more of the examples herein, wherein the one ormore processors are configured to implement a prediction phase to atleast one of i) detect plaque type, ii) classify anatomical severity ofvessel blockage, and/or iii) classify a hemodynamic severity of vesselobstructions within an unseen portion of the volumetric image data set.

Example 19

The system of one or more of the examples herein, wherein the machinelearning-based VOA model generates a sequence of cubes from the MPRimage, each of the cubes including a group of voxel from the MPR image,the sequence of cubes created within sections of the VOI resulting in asequence of cubes for corresponding sections.

Example 20

The system of one or more of the examples herein, wherein the machinelearning-based VOA model extracts image features associated with cubesfrom the sequence of cubes independently.

Example 21

The system of one or more of the examples herein, wherein the machinelearning analyzes the image features in sequential dependence.

Example 22

The system of one or more of the examples herein, wherein a size of thecube is defined to contain a whole lumen for the VOI and a portion oftissue outside of the lumen to facilitate extracting the VOI parameterin connection with positive remodeling, wherein positive remodelingrefers to a direction of atherosclerotic plaque growth.

Example 23

The system of one or more of the examples herein, wherein the axialtrajectory corresponds to a coronary centerline of the VOI, the coronarycenterline representing a center of a coronary lumen along a coronarysection of interest, the axial trajectory may correspond to a singlecoronary artery, a coronary bifurcation or a full coronary tree,wherein, when the coronary section of interest includes one or morebifurcation(s), the coronary centerline includes the one or morebifurcations.

Example 24

The system of one or more of the examples herein, wherein the machinelearning-based VOA model is based on a recurrent convolutional neuralnetwork (RCNN) which is employed to analyze a vicinity along the axialtrajectory of the VOI in the MPR image, the RCNN connects aconvolutional neural network (CNN) with a recurrent neural network (RNN)connected in series to analyze the portion of the MPR along the axialtrajectory as a sequential input.

Example 25

The system of one or more of the examples herein, wherein the machinelearning-based VOA model applies at least one convolution layer followedby a max pooling layer to extract an image feature of interest from theMPR image and utilizes classifiers for at least one of detecting plaquetype, characterizing plaque type, detecting stenosis or determining ananatomical significance of a stenosis.

Example 26

The system of one or more of the examples herein, wherein the machinelearning-based VOA model includes a feature extraction for creating afeature vector based on the MPR image, the feature vector comprises aseries of factors that are measured or extracted from a referencedatabase of images, the series of factors describing or characterizingthe nature of a corresponding wall region of the vessel of interest, themachine learning-based VOA model further including a classifier toclassify the feature vector extracted from the MPR image.

Example 27

The system of one or more of the examples herein, wherein the VOIparameter includes at least one of coronary plaque type, anatomicalcoronary lesion severity or functionally significant coronary lesionseverity, and wherein the machine learning-based VOA model assesses atleast one of i) functionally significant coronary lesion severity, ii)plaque type or iii) anatomical coronary lesion severity.

Example 28

A system to train a vessel obstruction assessment (VOA) model,comprising:

memory configured to store a training database that includes volumetricimaging datasets for multiple patients and corresponding coronary arterydisease (CAD) related reference values, the volumetric image data setsbeing for a target organ that includes a vessel of interest, the CADrelated reference values corresponding to one or more points along avessel of interest within the corresponding imaging data set; and

one or more processors that, when executing program instructions storedin the memory, are configured to:

-   -   for at least a portion of the volumetric image data sets and        corresponding CAD related reference values,        -   extract an axial trajectory extending along of a vessel of            interest (VOI) within the corresponding volumetric image            dataset,        -   create a three-dimensional (3D) multi-planer reformatted            (MPR) image based on the corresponding volumetric image            dataset and the axial trajectory of the VOI, the MPR image            extending along the axial trajectory of the VOI, and        -   train a machine learning-based vessel obstruction assessment            (VOA) model based on the MPR images, which involves            extracting, from the MPR images, features characterizing a            CAD related parameter along the axial trajectory within the            VOI.

Example 29

The system of one or more of the examples herein, wherein the one ormore processors are further configured to align the CAD relatedreference values to spatial coordinates of the corresponding MPR images.

Example 30

The system of one or more of the examples herein, wherein the one ormore processors are further configured to generate a sequence of cubesfrom the corresponding MPR images, each of the cubes including a groupof voxel from the corresponding MPR image, the sequence of cubes createdwithin sections of the VOI resulting in a sequence of cubes forcorresponding sections.

Example 31

The system of one or more of the examples herein, wherein the one ormore processors are further configured to apply a convolutional neuralnetwork to a sequence of cubes alone the MPR image to build the machinelearning-based VOA model.

Example 32

The system of one or more of the examples herein, wherein the one ormore processors are further configured to generate a set of encodings atpoints along the axial trajectory to form a set of one-dimensional (1D)sequences, each of the 1D sequences representing a specific encodingalong the VOI, and to perform the training by applying a supervisedclassifier to learn a fractional flow reserve (FFR) classifier based onthe 1D sequences.

While particular embodiments of the present application have beendescribed, it is not intended that the present application be limitedthereto, as it is intended that the present application be as broad inscope as the art will allow and that the specification be read likewise.

For example, multi-phase CCTA datasets can be used, functionalassessment of renal arties in relation to the perfused kidney can beassess based on the methodology disclosed, the data processingoperations can be performed offline on images stored in digital storage,such as a PACS or VNA in DICOM (Digital Imaging and Communications inMedicine) format commonly used in the medical imaging arts. It willtherefore be appreciated by those skilled in the art that yet othermodifications could be made to the provided application withoutdeviating from its spirit and scope as claimed.

The embodiments described herein may include a variety of data storesand other memory and storage media as discussed above. These can residein a variety of locations, such as on a storage medium local to (and/orresident in) one or more of the computers or remote from any or all ofthe computers across the network. In a particular set of embodiments,the information may reside in a storage-area network (“SAN”) familiar tothose skilled in the art.

Similarly, any necessary files for performing the functions attributedto the computers, servers or other network devices may be stored locallyand/or remotely, as appropriate.

Where a system includes computerized devices, each such device caninclude hardware elements that may be electrically coupled via a bus,the elements including, for example, at least one central processingunit (“CPU” or “processor”), at least one input device (e.g., a mouse,keyboard, controller, touch screen or keypad) and at least one outputdevice (e.g., a display device, printer or speaker). Such a system mayalso include one or more storage devices, such as disk drives, opticalstorage devices and solid-state storage devices such as random accessmemory (“RAM”) or read-only memory (“ROM”), as well as removable mediadevices, memory cards, flash cards, etc.

Such devices also can include a computer-readable storage media reader,a communications device (e.g., a modem, a network card (wireless orwired), an infrared communication device, etc.) and working memory asdescribed above.

The computer-readable storage media reader can be connected with, orconfigured to receive, a computer-readable storage medium, representingremote, local, fixed and/or removable storage devices as well as storagemedia for temporarily and/or more permanently containing, storing,transmitting and retrieving computer-readable information. The systemand various devices also typically will include a number of softwareapplications, modules, services, or other elements located within atleast one working memory device, including an operating system andapplication programs, such as a client application or web browser.

It should be appreciated that alternate embodiments may have numerousvariations from that described above. For example, customized hardwaremight also be used and/or particular elements might be implemented inhardware, software (including portable software, such as applets) orboth.

Further, connection to other computing devices such as networkinput/output devices may be employed.

Various embodiments may further include receiving, sending, or storinginstructions and/or data implemented in accordance with the foregoingdescription upon a computer-readable medium. Storage media and computerreadable media for containing code, or portions of code, can include anyappropriate media known or used in the art, including storage media andcommunication media, such as, but not limited to, volatile andnon-volatile, removable and non-removable media implemented in anymethod or technology for storage and/or transmission of information suchas computer readable instructions, data structures, program modules orother data, including RAM, ROM, Electrically Erasable ProgrammableRead-Only Memory (“EEPROM”), flash memory or other memory technology,Compact Disc Read-Only Memory (“CD-ROM”), digital versatile disk (DVD)or other optical storage, magnetic cassettes, magnetic tape, magneticdisk storage or other magnetic storage devices or any other medium whichcan be used to store the desired information and which can be accessedby the system device. Based on the disclosure and teachings providedherein, a person of ordinary skill in the art will appreciate other waysand/or methods to implement the various embodiments.

The specification and drawings are, accordingly, to be regarded in anillustrative rather than a restrictive sense. It will, however, beevident that various modifications and changes may be made thereuntowithout departing from the broader spirit and scope of the presentapplication as set forth in the claims.

Other variations are within the spirit of the present disclosure. Thus,while the disclosed techniques are susceptible to various modificationsand alternative constructions, certain illustrated embodiments thereofare shown in the drawings and have been described above in detail. Itshould be understood, however, that there is no intention to limit thepresent application to the specific form or forms disclosed, but on thecontrary, the intention is to cover all modifications, alternativeconstructions and equivalents falling within the spirit and scope of thepresent application, as defined in the appended claims.

The use of the terms “a” and “an” and “the” and similar referents in thecontext of describing the disclosed embodiments (especially in thecontext of the following claims) are to be construed to cover both thesingular and the plural, unless otherwise indicated herein or clearlycontradicted by context. The terms “comprising,” “having,” “including”and “containing” are to be construed as open ended terms (i.e., meaning“including, but not limited to,”) unless otherwise noted. The term“connected,” when unmodified and referring to physical connections, isto be construed as partly or wholly contained within, attached to orjoined together, even if there is something intervening.

Recitation of ranges of values herein are merely intended to serve as ashorthand method of referring individually to each separate valuefalling within the range, unless otherwise indicated herein and eachseparate value is incorporated into the specification as if it wereindividually recited herein. The use of the term “set” (e.g., “a set ofitems”) or “subset” unless otherwise noted or contradicted by context,is to be construed as a nonempty collection comprising one or moremembers.

Further, unless otherwise noted or contradicted by context, the term“subset” of a corresponding set does not necessarily denote a propersubset of the corresponding set, but the subset and the correspondingset may be equal.

Operations of processes described herein can be performed in anysuitable order unless otherwise indicated herein or otherwise clearlycontradicted by context. Processes described herein (or variationsand/or combinations thereof) may be performed under the control of oneor more computer systems configured with executable instructions and maybe implemented as code (e.g., executable instructions, one or morecomputer programs or one or more applications) executing collectively onone or more processors, by hardware or combinations thereof. The codemay be stored on a computer-readable storage medium, for example, in theform of a computer program comprising a plurality of instructionsexecutable by one or more processors. The computer-readable storagemedium may be non-transitory.

Preferred embodiments of this disclosure are described herein, includingthe best mode known to the inventors for carrying out the presentapplication. Variations of those preferred embodiments may becomeapparent to those of ordinary skill in the art upon reading theforegoing description. The inventors expect skilled artisans to employsuch variations as appropriate and the inventors intend for embodimentsof the present disclosure to be practiced otherwise than as specificallydescribed herein.

Accordingly, the scope of the present disclosure includes allmodifications and equivalents of the subject matter recited in theclaims appended hereto as permitted by applicable law. Moreover, anycombination of the above-described elements in all possible variationsthereof is encompassed by the scope of the present disclosure unlessotherwise indicated herein or otherwise clearly contradicted by context.

All references, including publications, patent applications and patents,cited herein are hereby incorporated by reference to the same extent asif each reference were individually and specifically indicated to beincorporated by reference and were set forth in its entirety herein.

1. A method for assessing a vessel obstruction, comprising: a) obtaininga volumetric image dataset of a myocardium and at least one coronaryvessel, wherein the myocardium comprises muscular tissue of the heart;b) creating a three-dimensional (3D) image corresponding to a coronaryvessel of interest based on the volumetric image dataset of a); c)generating feature data that represents features of both the myocardiumand the coronary vessel of interest, wherein at least some of thefeature data is determined by a first machine learning-based model basedon the 3D image of b); and d) employing a second machine learning-basedmodel to determine at least one parameter based on the feature data ofc), wherein the at least one parameter represents functionallysignificant coronary lesion severity of the coronary vessel of interest.2. The method of claim 1, wherein: the at least one parameter thatrepresents functionally significant coronary lesion severity is selectedfrom the group consisting of: a fractional flow reserve (FFR) value,coronary flow reserve value, instantaneous wave-free ratio, hyperemicmyocardium perfusion value, index of microcirculatory resistance,pressure drop along a coronary artery, FFR along a coronary artery, andcombinations thereof.
 3. The method of claim 1, wherein: the creation ofthe 3D image is based on axial trajectory of the coronary vessel ofinterest that is extracted from the volumetric image dataset of a). 4.The method of claim 3, wherein the feature data of c) is generated by:using the axial trajectory of the coronary vessel of interest to sampleimage data that corresponds to both the myocardium and the coronaryvessel of interest; and using the sampled image data as input to thefirst machine learning-based model, wherein the first machinelearning-based model is configured to automatically generate the featuredata of c).
 5. The method of claim 3, wherein the feature data of c) isgenerated by: using the axial trajectory of the coronary vessel ofinterest to sample image data that corresponds to the coronary vessel ofinterest; using the image data as input to the first machinelearning-based model, wherein the first machine learning-based model isconfigured to automatically generate first feature data, wherein thefirst feature data represents features of the coronary vessel ofinterest; generating second feature data from additional myocardiumanalysis of the volumetric image dataset, wherein the second featuredata represents features of the myocardium; and combining the firstfeature data and the second feature data to generate the feature data ofc).
 6. The method of claim 3, wherein: the second feature data isdetermined using at least one of: a convolutional auto-encoder, Gaussianfilters, transmural perfusion ratio, Haralick features, myocardiumthickness, and heart shape.
 7. The method of claim 3, wherein: thesecond feature data is determined using an unsupervised machine learningwhich employs a convolution auto-encoder.
 8. The method of claim 3,further comprising: repeating the operations of c) and d) using imagedata corresponding to the axial trajectory of the coronary vessel ofinterest to determine values of the at least one parameter representingfunctionally significant coronary lesion severity along the length ofthe coronary vessel of interest.
 9. The method of claim 8, wherein: theimage data corresponding to the axial trajectory of the coronary vesselincludes cubes or cuboids that are sampled from the 3D image and extendalong the axial trajectory of the coronary vessel of interest.
 10. Themethod of claim 3, wherein: the axial trajectory of the coronary vesselof interest corresponds to a centerline of the coronary vessel ofinterest.
 11. The method of claim 3, wherein: the 3D image comprises amulti-planer reformatted (MPR) image that is constructed from samples ofthe volumetric image dataset along an axial trajectory of the coronaryvessel of interest.
 12. The method of claim 1, wherein: multiple 3Dimages are created based on the volumetric dataset, wherein the multiple3D images correspond to a plurality of coronary vessels of interest andare based on extraction of axial trajectories for the plurality ofcoronary vessels of interest.
 13. The method of claim 12, wherein: theaxial trajectories for the plurality of coronary vessels of interestcorresponds to centerlines for plurality of coronary vessels ofinterest.
 14. The method of claim 12, further comprising: using sampledimage data from the multiple 3D images as input to the first machinelearning-based model to generate data representing features of theplurality of coronary vessels of interest, wherein such data is includedin the feature data of c).
 15. The method of claim 1, wherein: the firstmachine learning-based model comprises a recurrent convolutional neuralnetwork or at least one convolutional autoencoder.
 16. The method ofclaim 1, wherein: the second machine learning-based model comprises asupervised machine learning classifier.
 17. The method of claim 16,wherein: the second machine learning-based model comprises a neuralnetwork, at least one SoftMax classifier, or a support vector machine.18. The method of claim 1, wherein: the second machine learning-basedmodel is trained using reference values derived from invasivemeasurements involving pullback of a guidewire located in a targetcoronary vessel, measured coronary flow reserve, index ofmicrocirculatory resistance, major adverse cardiac events orrevascularization after acquisition of the volume image data set,results of cardiac stress test, and results of myocardial imaging bymagnetic resonance imaging (MRI), SPECT, PET, CT perfusion, orultrasound.
 19. The method of claim 18, wherein: the training of thesecond machine learning-based model involves aligning the referencevalues to spatial coordinates of a 3D image corresponding to the targetcoronary vessel.
 20. A system for assessing a vessel obstruction,comprising: memory configured to store program instructions and to storea volumetric image dataset of a myocardium and at least one coronaryvessel, wherein the myocardium comprises muscular tissue of the heart;at least one processor that, when executing the program instructionsstored in the memory, is configured to: a) obtain a volumetric imagedataset of a myocardium and at least one coronary vessel; b) create athree-dimensional (3D) image corresponding to a coronary vessel ofinterest based on the volumetric image dataset of a); c) generatefeature data that represents features of both the myocardium and thecoronary vessel of interest, wherein at least some of the feature datais determined by a first machine learning-based model based on the 3Dimage of b); and d) employ a second machine learning-based model todetermine at least one parameter based on the feature data of c),wherein the at least one parameter represents functionally significantcoronary lesion severity of the coronary vessel of interest.
 21. Thesystem of claim 20, wherein: the at least one parameter that representsfunctionally significant coronary lesion severity is selected from thegroup consisting of: a fractional flow reserve (FFR) value, coronaryflow reserve value, instantaneous wave-free ratio, hyperemic myocardiumperfusion value, index of microcirculatory resistance, pressure dropalong a coronary artery, FFR along a coronary artery, and combinationsthereof.
 22. The system of claim 20, wherein: the at least one processoris further configured to extract an axial trajectory extending along ofthe coronary vessel of interest within the volumetric image dataset. 23.The system of claim 22, wherein: the creation of the 3D image is furtherbased on the volumetric image dataset and the axial trajectory of thecoronary vessel of interest.
 24. The system of claim 22, wherein: the 3Dimage comprises a multi-planer reformatted (MPR) image that isconstructed from samples of the volumetric image dataset along the axialtrajectory of the coronary vessel of interest.
 25. The system of claim22, wherein: the axial trajectory of the coronary vessel of interestcorresponds to a centerline of the coronary vessel of interest.
 26. Thesystem of claim 22, wherein: the at least one processor is configured torepeat the operations of c) and d) using image data corresponding to theaxial trajectory of the coronary vessel of interest to determine valuesof the at least one parameter representing functionally significantcoronary lesion severity along the length of the coronary vessel ofinterest.
 27. The system of claim 22, wherein: the firstmachine-learning based model is configured to automatically extractimage features associated with cubes or cuboids that are sampled fromthe 3D image and extend along the axial trajectory of the coronaryvessel of interest.
 28. The system of claim 20, wherein: the at leastone processor is configured to create multiple 3D images based on thevolumetric dataset, wherein the multiple 3D images correspond to aplurality of coronary vessels of interest and are based on extraction ofaxial trajectories for the plurality of coronary vessels of interest.29. The system of claim 28, wherein: the axial trajectories for theplurality of coronary vessels of interest corresponds to centerlines forplurality of coronary vessels of interest.
 30. The system of claim 28,wherein: the at least one processor is configured to use sampled imagedata from the multiple 3D images as input to the first machinelearning-based model to generate data representing features of theplurality of coronary vessels of interest, wherein such data is includedin the feature data of c).
 31. The system of claim 20, wherein: thefirst machine learning-based model comprises a recurrent convolutionalneural network or at least one convolutional autoencoder.
 32. The systemof claim 20, wherein: the second machine learning-based model comprisesa supervised machine learning classifier.
 33. The system of claim 32,wherein: the second machine learning-based model comprises a neuralnetwork, at least one SoftMax classifier, or a support vector machine.34. The system of claim 20, wherein: the second machine learning-basedmodel is trained using reference values derived from invasivemeasurements involving pullback of a guidewire located in a targetcoronary vessel.
 35. The system of claim 34, wherein: the second machinelearning-based model is trained by aligning the reference values tospatial coordinates of a 3D image corresponding to the target coronaryvessel.